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Overview 


Young adults with histories of foster care or juvenile justice custody often experience poor outcomes 
across a number of domains, on average, relative to their peers. While government funding for 
services targeting these groups of young people has increased in recent years, research on the 
effectiveness of such services is limited, and few of the programs that have been rigorously tested 
have been found to improve outcomes. 

The Youth Villages Transitional Living Evaluation tested whether the Transitional Living program, 
operated by the social service organization Y outh Villages, makes a difference in the lives of young 
men and women with histories of foster care or juvenile justice custody. The program, which was 
renamed “YVLifeSet” in April 2015, is intended to help these young people make a successful 
transition to adulthood by providing intensive, individualized, and clinically focused case manage- 
ment, support, and counseling. 

The evaluation used a rigorous random assignment design and was set in Tennessee, where Youth 
Villages operates its largest Transitional Living program. Lrom October 2010 to October 2012, more 
than 1,300 young people were assigned, at random, to either a program group, which was offered the 
Transitional Living program’s services, or to a control group, which was not offered those services. 
Using survey and administrative data, the evaluation team measured outcomes for both groups over 
time to assess whether Transitional Living services led to better outcomes for the program group 
compared with the control group’s outcomes. 

This is the third major report in the evaluation. The first report provides a detailed description of the 
Transitional Living program model and assesses its implementation. The second report assesses 
whether the program improved key outcomes during the first year after young people were enrolled 
in the study. That report relies largely on survey data to analyze the program’s impacts in the six 
domains that it was designed to affect: education; employment and earnings; housing stability and 
economic well-being; social support; health and safety; and criminal involvement. This third report 
uses administrative data to assess the program’s impacts in three of the original six domains — 
education; employment and earnings; and criminal involvement — during the second year after 
study enrollment. Taken together, the one- and two-year results show that participation in the 
Transitional Living program had modest, positive impacts on a broad range of outcomes. The 
program boosted earnings, increased housing stability and economic well-being, and improved some 
outcomes related to health and safety. However, it did not improve outcomes in the areas of educa- 
tion, social support, or criminal involvement. 

These results indicate that the Transitional Living program can improve multiple outcomes for 
young adults with histories of foster care or juvenile justice custody, a notable finding given how 
few other programs that serve these populations have been shown to have an effect. As a next step, 
Youth Villages aims to build on the areas where the program has already been successful by testing 
modifications to the YVLifeSet model; the hope is that such modifications will further improve 
young people’s outcomes, particularly in domains where the program has not yet produced positive 
impacts. 
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Preface 


The transition from adolescence to adulthood is a critical and often challenging time, especially 
for young people who have been in the foster care or juvenile justice system. These individuals 
face difficulties that are much less common among their peers with no history of state custody, 
such as low levels of education, minimal formal work history, mental health and substance 
abuse problems, weak social support, extreme poverty, and housing instability. While others 
their age frequently get help from their parents well into their twenties, young people who are 
leaving state custody tend to have relatively little financial, emotional, or social support. 
Moreover, many of them suffer from the lingering effects of childhood trauma and the inade- 
quacies of the government systems that acted as their guardians. Given such circumstances, it is 
not surprising that these young people often struggle in many areas as they enter adulthood. 

One program designed to help them is the YVLifeSet program — formerly the Youth Villages 
Transitional Living program — which offers intensive case management, support, and counsel- 
ing on issues related to housing, employment, education, life skills, and behavioral health to 
young people who were formerly in foster care or juvenile justice custody. Taken together, the 
one- and two-year results of the Youth Villages Transitional Living Evaluation, conducted in 
the state of Tennessee, show that the program can make positive differences in the lives of 
young adults who were in foster care or juvenile justice custody as teenagers. The young people 
who were offered the program’s services had improved earnings, experienced less homeless- 
ness and material hardship, and had better mental health than those who were not offered its 
services. However, while the program helped to stabilize many of its participants as they made 
the transition to adulthood, it did not have an impact on longer-term outcomes, such as educa- 
tional attainment. 

As an organization committed to continuous learning and program improvement, Youth 
Villages is working to strengthen the YVLifeSet model by testing new strategies to enhance its 
positive effects on the young people it serves — using risk assessments to better tailor services 
to participants, increasing efforts to help young people who are at high risk of criminal justice 
involvement, and ramping up its technological capabilities to improve data collection and 
service delivery. 

Given its early success, it is critical that we continue to leam with the YVLifeSet program as it 
expands to additional states and implements program improvement strategies and other services 
for young people who lack strong family supports and life skills. 

Gordon L. Berlin 

President, MDRC 


IX 



Acknowledgments 


This report would not have been possible without the support of many individuals and organiza- 
tions. In particular, the Youth Villages Transitional Living Evaluation and the production of this 
report were funded by the Edna McConnell Clark Foundation, the Annie E. Casey Foundation, 
and the Bill & Melinda Gates Foundation. We would also like to thank Mark Courtney (Univer- 
sity of Chicago) for his central role as MDRC’s research partner throughout this evaluation. 

We are tremendously grateful to several individuals on the staff at Youth Villages, in- 
cluding Sarah Hurley, Melanie Manns, Tim Goldsmith, Kristin Landers, and Pat Lawler, with 
whom we enjoyed a productive collaboration throughout this project. Sarah Hurley, Tim 
Goldsmith, and Kristin Landers carefully reviewed earlier drafts of the report and provided 
insightful feedback. Sarah Hurley also worked closely with MDRC to conceptualize and launch 
the evaluation, facilitate our communication with staff at Youth Villages and partner agencies, 
supply program participation data, and provide feedback at every stage of the research, among 
many other efforts that made this evaluation possible. Kristin Landers met with MDRC staff to 
help us understand the Transitional Living model and interpret program data and procedures. 
Melanie Manns monitored study enrollment, tracked participant samples, organized study 
paperwork for MDRC, and generally kept track of research activities onsite. 

We accessed employment and earnings data with the help of Carl Attkisson at the 
Tennessee Department of Labor and Workforce Development. 

At MDRC, John Martinez developed this project from the beginning and was the pro- 
ject director throughout. Sara Muller-Ravett and Joseph Broadus made sure random assignment 
and onsite operations went smoothly. John Martinez, Dan Bloom, Virginia Knox, Michelle 
Manno, Chuck Michalopoulos, Alice Tufel, and Mark Courtney provided thoughtful comments 
on several drafts of this report. Michelle Manno and Julianna Alson conducted the implementa- 
tion research. Brit Henderson processed the postsecondary education and criminal justice data 
used in this report, while Danielle Cummings processed the employment and earnings data and 
conducted the impact analysis. Timothy Rudd, Yana Kusayeva, and David H. Greenberg 
(University of Maryland, Baltimore County) completed the cost analysis for the project. Dannia 
Guzman coordinated the production of the report. Carole Campbell edited the report, and An n 
Kottner prepared it for publication. 

We are especially grateful to the young people who participated in the study. They en- 
thusiastically participated in surveys, interviews, and focus groups and allowed us to leam from 
their experiences. Many of the study participants were excited to help provide knowledge that 
could lead to better services for other young people in similar situations. We hope that this 
report will fulfill that wish. 

The Authors 


XI 



Executive Summary 


Large numbers of young people in the United States were in foster care or in juvenile justice 
custody as teenagers, and many of them have a difficult time making a successful transition to 
independent adulthood as they leave these systems. Most of them faced a number of disad- 
vantages during childhood and often have poor outcomes across several domains relative to their 
peers as they become adults. While government funding to help these groups has increased, few 
of the programs that have been rigorously evaluated have been found to improve outcomes. 

To advance knowledge in this area, the Youth Villages program sought an independent 
evaluation of its Transitional Living program — now known as “YVLifeSef ’ — which is one 
example of an “independent living” program. 1 The Transitional Living program aims to help 
young men and women make the transition to adulthood by providing intensive, individualized, 
and clinically focused case management, support, and counseling. The evaluation used a 
rigorous random assignment design in which study sample members were assigned at random 
to either a program group that was offered the Transitional Living program services or to a 
control group that was not offered those services. The Edna McConnell Clark Foundation, the 
Annie E. Casey Foundation, and the Bill & Melinda Gates Foundation funded the evaluation, 
which was led by MDRC in concert with Mark Courtney of the University of Chicago. 

This third major report in the evaluation builds on the one-year findings and assesses 
the estimated two-year impacts of the Transitional Living program using administrative data for 
three outcome domains: education, employment and earnings, and criminal involvement. The 
report also presents information on the costs of operating the Transitional Living program. 

Key findings overall and from the two-year analysis include: 

• Transitional Living did not increase young people’s average earnings during 
the second year of follow-up, but it had a modest, positive effect at some 
earnings levels during this time period and it led to modest increases in em- 
ployment and earnings over the full two-year study period. 

• Statistically significant effects were not observed in Year 1 in the education, 
social support, and criminal involvement domains, and did not emerge in 
Year 2 (though social support was measured in Year 1 only). 


'The Transitional Living program was renamed “YVLifeSet” in April 2015. Because the name did not 
change until after the study period ended, this report refers to the program as “Transitional Living.” 
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• The program increased housing stability and economic well-being and im- 
proved some of the primary outcomes related to health and safety in Year 1, 
but data were not available to assess whether impacts in these domains con- 
tinued into Year 2. 

As noted above, this report does not include two-year results for three domains that were 
included in the one-year analysis — housing stability and economic well-being, social support, 
and health and safety — because administrative data for those domains do not exist, are difficult 
to obtain, or do not fully measure relevant outcomes. Therefore, this report provides only a 
partial picture of the two-year impacts of the Transitional Living program, particularly since the 
one-year analysis showed significant impacts on outcomes in two of the excluded domains. 


Background and Policy Context 

For those who have spent time in the foster care or juvenile justice system, or both, the transition 
from adolescence to adulthood can be particularly challenging. Such young people often contend 
with low levels of educational attainment, minimal formal work experience, mental health and 
substance use problems, weak social support, extreme poverty, and housing instability. 

Recent federal legislation has increased the funding of services for young people who 
are aging out of foster care. The John Chafee Foster Care Independence Act of 1999 gave states 
more funding to support independent living services, room and board, and Medicaid for young 
people in foster care up to age 21. 2 The subsequent Fostering Connections to Success and 
Increasing Adoptions Act of 2008 provided funding for states to extend foster care from age 1 8 
through age 21 for eligible young people and to further expand independent living services. 
However, the availability and extent of these services vary widely by state depending on 
whether and how states choose to take advantage of federal funds. 

Services for young people who are leaving juvenile justice placements have not been 
funded as consistently as services for those leaving foster care, though some young people who 
have been in juvenile justice custody are eligible for services funded by the Chafee and Foster- 
ing Connections acts. In addition, a general focus on “reentry” services for adults leaving prison 
and jail has led to federal funding to serve young people with a juvenile justice history. 


2 Medicaid provisions under the Chafee Act have now been superseded by those of the Affordable Care 
Act, under which all young people in foster care on their 1 8th birthday are eligible for Medicaid up to age 26. 
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The Transitional Living Program 

The Transitional Living program is operated by Youth Villages, a nonprofit social service 
organization based in Memphis, Tennessee. The organization operates a variety of residential 
and community-based programs serving more than 20,000 young people each year in 12 states 
and the District of Columbia. 

Transitional Living program services are expected to last an average of nine months. 
The program starts with assessments and the development of an individualized treatment plan 
that takes into account each participant’s particular needs and goals. The bulk of the services are 
then provided during weekly, hour-long Transitional Living sessions with a “TL Specialist,” 
who typically serves only eight young people at a time. 

The content of the Transitional Living sessions varies depending on each participant’s 
needs, but TL Specialists are expected to use evidence-infonned tools, counseling, and action- 
oriented activities. Evidence-infonned tools include specific curricula that cover topics like 
money management and job-seeking skills, as well as behavioral treatment strategies aimed at, 
for instance, helping participants overcome substance abuse problems. Counseling involves 
discussions between each participant and TL Specialist to address problems that may be 
impeding the young person’s progress toward stated goals. Finally, TL Specialists use action- 
oriented activities, such as taking a participant to a bank to open an account or to a community 
college to gather information about classes. 

In addition, trauma- focused cognitive behavioral therapy, a 12- to 20-week course of 
therapy offered by specially trained Youth Villages staff, is provided if it is clinically indicated. 
TL Specialists may also refer participants to other services in the community, such as General 
Educational Development (GED) classes, specialized mental health services, or housing 
services. TL Specialists have access to some flexible funds to support those who need money 
for expenses such as purchasing appropriate clothing for interviews or an apartment application 
fee. They also encourage young people to participate in monthly group social and learning 
activities with others in the program. Finally, educationaFvocational coordinators are available 
to provide extra support to young people who want to go to college, take vocational training, or 
find a job. 


The Youth Villages Transitional Living Evaluation 

The Youth Villages Transitional Living Evaluation assessed the impacts of the Transitional 
Living program in Tennessee. The study sample includes men and women ages 18 to 24 who 
were living in Tennessee and who had left foster care or juvenile justice custody as teenagers or 
were aging out at 18. Between October 2010 and October 2012, 1,322 young people were 
assigned at random to either a program group, whose members were offered Transitional Living 
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program services, or a control group, whose members were not offered Transitional Living 
program services, but were provided with a list of other social service resources that were 
available in the community. 

By measuring outcomes for both groups over time, the research team could assess 
whether Transitional Living services led to better outcomes for the program group. Owing to 
the random assignment design, the research groups were expected to be comparable on both 
measured and umneasured characteristics when the study began. Therefore, statistically signifi- 
cant differences in outcomes that emerge between the two groups can be attributed with some 
confidence to the offer of Transitional Living services to the program group. These differences 
in outcomes are considered “impacts” or “effects” of the Transitional Living program. 

Similar to other young people with histories of foster care or juvenile justice custody, 
those who enrolled in the study averaged relatively low levels of educational attainment and 
employment at study entry, while experiencing relatively high rates of arrest and housing 
instability. They are diverse in terms of gender and race: 48 percent of the sample are women, 
over 50 percent are white/non-Hispanic, and a substantial minority are black/non-Hispanic (37 
percent). Finally, they come from varied custody backgrounds, with their first custody place- 
ment tending to occur in their teens. Sixty-one percent of the sample reported having been in 
custody because they had been neglected or abused (foster care), while 52 percent indicated that 
they had been in custody for delinquency (juvenile justice). About 13 percent of the study 
sample had experienced both types of custody. 


Program Implementation and Service Receipt Differences 
Between the Two Research Groups 

The Transitional Living program was implemented largely in accordance with the program 
model. A substantial portion of the program group received services at the expected average 
dosage (level and intensity) of the Transitional Living program model. About two-thirds 
participated in program services for at least five months and about half participated for at least 
nine months. Nearly all program group members participated in at least one Transitional Living 
session. While involved in the program, each individual participated in nearly one session per 
week, averaging over an hour per session. During these sessions, TL Specialists and participants 
covered a wide range of issues, with education, employment, and housing discussed most often. 

Overall, there are large, statistically significant differences between the program and 
control groups in the dosage of the services they received. The program group was more likely 
than the control group to have had a case manager or social worker (75 percent compared with 
44 percent) and to have met with that person at least once a week (60 percent compared with 20 
percent). They were also more likely to have received help, from any source, with problems 
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related to education, employment, finances, housing, and daily living. However, while there 
was a clear difference in the level of services received, many control group members also 
obtained case management and other services. 


One-Year Impacts of the Transitional Living Program 

The primary source of outcome data for the one-year impact report is a survey that was admin- 
istered by NORC at the University of Chicago to all sample members one year after they 
entered the study. Outcomes in six key domains were covered: education; employment and 
earnings; housing stability and economic well-being; social support; health and safety; and 
criminal involvement. 

Statistically significant impacts on primary outcomes were detected in employment and 
earnings, housing stability and economic well-being, and health and safety. The program led to 
an increase of over $600 in earnings in the year before the survey interview, driven, at least in 
part, by an increase in the percentage of young people who were employed. Program group 
members experienced significantly fewer types of housing instability than control group 
members, most notably reductions in homelessness and “couch surfing” (staying in someone 
else’s home temporarily when not having a pennanent place to live). Similarly, the Transitional 
Living program reduced the incidence of economic hardship, driven by decreases in the 
percentage of those who did not have necessary clothing or shoes and in the percentage of 
young people who had delayed paying a bill in order to buy food. Finally, the program im- 
proved mental health and reduced the percentage of those involved in violent relationships. 
However, it did not significantly affect other key measures of health and safety, including 
substance use, condom use, and victimization. No statistically significant effects were found on 
primary outcomes in education, social support, and criminal involvement. 

The research team also assessed differences in impacts across four sets of subgroup 
characteristics. In the one-year analysis, the research team found that the impacts of Transitional 
Living were consistent across these subgroups. 

Overall, while the statistically significant impacts detected at one year proved modest, 
their breadth across several domains is consistent with the highly individualized nature of the 
program model, which is designed to address the wide variety of needs and circumstances of the 
young people it serves. The one-year impact findings were promising, especially given the lack 
of statistically significant, positive impacts for other programs targeting similar populations. 
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Two-Year Impacts 

The two-year analysis estimates impacts in three domains: (1) education, using postsecondary 
enrollment data from the National Student Clearinghouse; (2) employment and earnings, using 
unemployment insurance data from the Tennessee Department of Labor and Workforce 
Development; and (3) criminal involvement, using arrest and conviction data from the Tennes- 
see Bureau of Investigation. For all three administrative data sources, the research team created 
Year 1, Year 2, and overall outcome measures and estimated impacts on these measures. With 
the exception of postsecondary enrollment data, Year 1 impacts on outcomes measured using 
administrative data were not assessed or presented in the one-year report. 

Impacts on Education 

As shown in Table ES.l, Transitional Living did not produce a statistically significant 
impact on the primary outcome in the education domain — enrollment in a postsecondary 
institution — either across the two years of follow-up or looking at each year individually. 
About one-fourth of both program and control group members enrolled in a postsecondary 
institution at some point in the two years following study enrollment; for both research groups, 
enrollment rates declined in Year 2 from the levels observed in Year 1 . 

Impacts on Employment and Earnings 

No statistically significant differences between the program and control groups were 
observed in average total earnings when administrative data were used (Table ES.l). Total 
earnings for both research groups hovered at about $5,000, with earnings increasing from about 
$2,000 in Year 1 to about $3,000 in Year 2. Overall, these earnings levels are quite low. 

While Transitional Living did not produce a statistically significant impact on young 
people’s average earnings, the program did have a modest, positive effect at some earnings 
levels. Focusing on Year 2, the Transitional Living program had a statistically significant 
impact of 6 percentage points on the proportion of young people earning $2,500 or more. 

Impacts on Criminal Involvement 

There are no statistically significant differences between program and control group 
members in their rates of arrest or conviction (Table ES. 1 ). Just under half of the members of both 
groups were arrested at some point in the two years following study enrollment. About one-third 
of sample members were arrested in Year 1, and one-third in Year 2. About one-fifth of both 
research groups were convicted of a crime during the two years following study enrollment. 
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Table ES.l 


Two-Year Impacts on Education, Employment and Earnings, 
and Criminal Involvement 


Primary Outcome, by Domain' 1 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Education 

Enrolled in a postsecondary institution (%) 

22.0 

25.4 

-3.4 

0.111 

Year 1 

18.9 

18.4 

0.4 

0.821 

Year 2 

11.4 

13.6 

-2.2 

0.212 

Employment and earnings 

Total earnings ($) 

5,240 

5,016 

224 

0.555 

Year 1 

2,233 

2,130 

103 

0.562 

Year 2 

3,006 

2,885 

121 

0.641 

Y ear 1 earnings (%) 

$2,500 or more 

30.5 

25.5 

5.1 ** 

0.035 

$5,000 or more 

16.7 

13.4 

3.3 * 

0.083 

$7,500 or more 

9.0 

7.9 

1.1 

0.462 

Y ear 2 earnings (%) 

$2,500 or more 

36.6 

30.3 

6.3 ** 

0.016 

$5,000 or more 

22.9 

19.2 

3.7 

0.105 

$7,500 or more 

14.2 

13.1 

1.1 

0.566 

Criminal involvement 

Arrested (%) 

47.7 

47.6 

0.1 

0.972 

Year 1 

30.7 

31.1 

-0.4 

0.877 

Year 2 

34.0 

34.9 

-0.9 

0.721 

Convicted of a crime (%) 

19.7 

17.7 

2.0 

0.350 

Year 1 

10.5 

9.5 

1.0 

0.564 

Year 2 

13.4 

12.6 

0.9 

0.635 

Sample size (total = 1,322) 

788 

534 




(continued) 
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Table ES.l continued 


SOURCE: MDRC calculations based on postsecondary education data from the National Student 
Clearinghouse, unemployment insurance quarterly earnings data from the Tennessee Department of 
Labor and Workforce Development, and criminal history data from the Tennessee Bureau of 
Investigation. 

NOTES: Results in this table are regression-adjusted, controlling for pre-random assignment 
characteristics. Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

Unemployment insurance data are quarter-based; thus, the two-year follow-up period in this table 
does not necessarily represent the 24 months following random assignment. Rather, the follow-up 
period represents the eight quarters following the quarter in which participants were randomly 
assigned. 


Impacts by Subgroups of Young People 

Subgroup analyses conducted at one year of follow-up examined the pattern of impacts 
on primary outcomes by history of juvenile justice custody; by geographic setting; by receipt of 
extended foster care services at baseline through Tennessee’s Department of Children’s Ser- 
vices; 3 and by latent classes, or clusters identified by an analysis meant to group young people 
based on their readiness for independent living using key baseline characteristics. As in the one- 
year analysis, the results of the two-year analysis showed that the impacts of the Transitional 
Living program are consistent across the four subgroups analyzed. 


Assessment of Two-Year Impact Findings 

The Transitional Living program maintained some modest effects on employment and earnings 
outcomes after two years, though evidence of the program’s effects in this domain are stronger 
for Year 1 than for Year 2. Notably, earnings levels based on administrative data were quite low 
for both research groups, underscoring the level of disadvantage experienced by young people 
in the study. 

Statistically significant improvements in education and criminal involvement did not 
emerge with longer-term follow-up for the outcomes that were measured in these domains. 
Many of these young people were likely in dire need of income or may have faced more urgent 
problems related to housing instability or personal safety that precluded their pursuit of higher 


3 The Fostering Connections to Success and Increasing Adoptions Act of 2008 provided funding for states to 
extend foster care from age 18 through 21 for eligible young people and to expand independent living services. 
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education. At the same time, much research has shown that it is very difficult to improve 
criminal involvement outcomes. Services related to criminal justice issues consisted of case 
management and counseling from TL Specialists, including some interventions related to risky 
behaviors. However, the program did not have additional components that explicitly focused on 
changing criminal behavior as it did for some other domains. 

Overall, the two-year findings are fairly consistent with those found after the first year 
of follow-up. However, the research team was unable to assess two-year impacts in three 
domains that were examined in the one-year analysis, including two domains in which statisti- 
cally significant effects were found. Specifically, it is unknown whether significant impacts on 
housing stability and economic well-being and health and safety persisted into a second year. 


Discussion and Policy Implications 

The Youth Villages Transitional Living Evaluation is one of the largest rigorous evaluations of 
services in the United States for young people who were formerly in the foster care or juvenile 
justice systems. The findings presented here have important implications for future policymak- 
ing and research. Taken together, the one- and two-year impact findings show that the Transi- 
tional Living program led to modest, positive impacts on a broad range of outcomes in three of 
the six domains that were measured. 

• Transitional Living led to modest increases in employment and earnings. The evi- 
dence for this finding is stronger for Year 1, showing a statistically significant impact on 
average earnings for that year, when survey data were used; however, in both years, the admin- 
istrative data analysis indicates that the program increased the proportion of young people 
earning over $2,500 per year, a threshold that falls between the average annual earnings of the 
study sample in Years 1 and 2. 

• The program increased housing stability and economic well-being, including a re- 
duction in homelessness, by one year after study enrollment. Data are not available to assess 
whether impacts in that domain continued into the second year. 

• Significant impacts were found on some of the primary outcomes related to health 

and safety, but it is not known whether these impacts continued beyond one year because data 
were not available. 

• Transitional Living did not lead to significant impacts on education, social support 
(measured only at one year), or criminal involvement. The program did not lead to increases 
in high school graduation or receipt of a GED certificate at one year, nor did it increase postsec- 
ondary enrollment over two years. Similarly, it did not reduce criminal involvement as meas- 
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ured by self-reporting at one year or as measured by administrative data over two years. Finally, 
there was not a significant impact on social support based on survey data at the one-year mark. 

The impact analysis suggests that the Transitional Living program was able to improve 
outcomes related to immediate needs, such as housing, food, clothing, and avoiding violent 
relationships, but was not as successful in affecting less immediate outcomes, such as educa- 
tional attainment. Given the challenges that these young people must overcome, it may be that 
addressing the basic needs of participants requires a good deal of staff time and resources. 
Accordingly, less time may be left to address other issues. Nevertheless, Transitional Living 
affected a broad range of outcomes in some very important domains for young people who are 
experiencing the transition to adulthood. These findings are particularly noteworthy, given how 
few other programs have been shown to improve outcomes among young adults formerly in 
foster care or juvenile justice custody. The results of this study provide evidence that interven- 
tions are available that can lessen some of the difficulties that many of these young people face. 


Next Steps 

As an organization that emphasizes continuous learning and program improvement, Youth 
Villages is focused on using the evaluation findings as a springboard to strengthen the YVLife- 
Set model, fonnerly known as Transitional Living. Youth Villages plans to test new strategies 
intended to enhance the program’s positive effects on the young people it serves, building on 
the areas where the program has already been successful. Beginning in late 2016, Youth 
Villages will implement and test a few key modifications to the YVLifeSet model to assess 
whether they hold promise at a larger scale. Youth Villages is also considering launching a 
second large-scale study of YVLifeSet in a different context, outside the state of Tennessee. 
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Introduction 


Large numbers of young people in the United States were in foster care or in juvenile justice 
custody as teenagers, and many of them have a difficult time making a successful transition to 
independent adulthood as they leave these systems. In 2014, more than 65,000 young people 
between 14 and 20 years of age left the foster care system in the United States, with roughly 
one-third exiting because they aged out of the system. 1 The juvenile justice system also has a 
broad reach, as over 50,000 young people are held in residential placement facilities at any point 
in time. 2 Young people who are leaving these systems, most of whom faced a number of 
disadvantages during childhood, commonly experience poor outcomes across a number of 
domains relative to their peers as they become adults. While government funding for services 
targeting these groups has increased, few of the programs that have been rigorously evaluated 
have been found to improve outcomes. 

To advance knowledge in this area, the Youth Villages program sought an independent 
evaluation of its Transitional Living program — now known as “YVLifeSet” — which is one 
example of an “independent living” program or a program intended to help vulnerable young 
people to become self-sufficient. Transitional Living is operated by Youth Villages, a nonprofit 
social service organization based in Memphis, Tennessee, that has served emotionally and 
behaviorally troubled boys and girls of all ages since 1986. 3 The Transitional Living program 
aims to help young men and women make the transition to adulthood by providing intensive, 
individualized, and clinically focused case management, support, and counseling. Young people 
who were enrolled in the evaluation included those living in Tennessee who were 18 to 24 years 
of age and had left foster care or juvenile justice custody as teenagers or were aging out of the 
system. 4 The evaluation used a rigorous random assignment design in which study sample 
members were assigned at random to either a program group that was offered the Transitional 
Living program services or to a control group that was not offered those services. The Edna 
McConnell Clark Foundation, the Annie E. Casey Foundation, and the Bill & Melinda Gates 
Foundation funded the evaluation, which was led by MDRC in concert with Mark Courtney of 
the University of Chicago. 


’This number refers to fiscal year 2014 (October 1, 2013, through September 30, 2014). See U.S. Depart- 
ment of Health and Human Services (2015). 

2 Office of Juvenile Justice and Delinquency Prevention (2015). 

3 The Transitional Living program was renamed “YVLifeSet” in April 2015. Because the name did not 
change until after the study period ended, this report refers to the program as “Transitional Living.” 

4 While this evaluation focuses on Transitional Living services provided to young people who have been in 
foster care or juvenile justice custody, the program also serves those who have not been in state custody but 
who could potentially benefit from such services. The program operates in seven states in addition to Tennes- 
see. 
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Background and Policy Context 

The transition from adolescence to adulthood is a critical and often trying time for young people 
of any background. For those who have spent time in state custody in the foster care or juvenile 
justice system, or both, this transition is often particularly challenging. Such young people 
often contend with low levels of educational attainment, minimal fomial work experience, 
mental health and substance use problems, weak social support, extreme poverty, and housing 
instability. 7 

Federal Legislation 

Recent federal legislation has increased the funding of services for young people who 
are aging out of foster care. The John Chafee Foster Care Independence Act of 1999 gave states 
more funding to support independent living services, room and board, and Medicaid for young 
people in foster care up to age 21. 8 The subsequent Fostering Connections to Success and 
Increasing Adoptions Act of 2008 provided funding for states to extend foster care from age 1 8 
through age 21 for eligible young people and to further expand independent living services. 
However, the availability and extent of these services vary widely by state depending on 
whether and how states choose to take advantage of federal funds. To date, nearly half of the 50 
states have used federal funding to extend foster care past age 1 8 (with states imposing varying 
requirements for young people to remain in care). 9 See Box 1 for more infomiation on how 
Temiessee, the state in which the Transitional Living Evaluation was carried out, has ap- 
proached its adoption of extended foster care services and has expanded independent living 
services under the provisions of the Fostering Connections Act. 

Services for young people who are leaving juvenile justice placements have not been funded as 
consistently as services for those who are leaving foster care, though some young people who 
have been in juvenile justice custody are eligible for services funded by the Chafee and Foster- 
ing Connections acts. In addition, a general focus on “reentry” services for adults leaving prison 
and jail has led to federal funding to serve young people with a juvenile justice history. For 
example, the Second Chance Act provides funds to government agencies and nonprofit organi- 
zations to offer employment assistance, substance abuse treatment, housing assistance, and 


7 Courtney, Piliavin, Grogan-Kaylor, and Nesmith (1998); Courtney (2009); Reilly (2003); Nellis and 
Wayman (2009); Sedlak and McPherson (2010). 

Medicaid provisions under the Chafee Act have now been superseded by those of the Affordable Care 
Act, under which all young people in foster care on their 1 8th birthday are eligible for Medicaid up to age 26. 

"’Unpublished data from the U.S. Department of Health and Human Services, Administration for Children 
and Families, Children’s Bureau (2016). 
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Box 1 

Extension or Re-Establishment of Foster Care Services 


Across the state of Tennessee, services other than Transitional Living were available through- 
out the two-year study follow-up period to young adults with histories of state custody. Some 
of these resources were available through the Tennessee Department of Children’s Services 
(DCS), which provided financial support and case management services. Until July 2012, 
these services were offered under the state’s Post-Custody Services program.* The primary 
Post-Custody Services included financial assistance for transportation, housing, education, and 
job-training; access to education and training vouchers for postsecondary school or vocational 
training; and twice-quarterly meetings with a case manager. To receive Post-Custody Services, 
young people were required to be (1) working toward their high school diploma or equivalen- 
cy or (2) enrolled in an approved institution that provides postsecondary education or voca- 
tional training. Those leaving secure juvenile justice facilities were not eligible. Additionally, 
those who had not yet graduated from high school but were on track to do so had the option of 
remaining in a supported foster care placement until the age of 19. 

Sample enrollment for the Transitional Living Evaluation spanned the shift, in July 2012, from 
the state’s Post-Custody Services program to Extension or Re-Establishment of Foster Care 
(EFC) Services. This shift came as federal funding became available for states to extend foster 
care through age 21 and expand independent living services via the Fostering Connections to 
Success and Increasing Adoptions Act of 2008. Services and eligibility criteria under EFC are 
similar to those under the Post-Custody Services program, with a few notable differences: 

1 . Eligibility criteria were expanded to include young people who have a serious physical 
or mental health condition that prevents them from pursuing education or full-time em- 
ployment. 

2. Y oung people who meet any of the three key eligibility criteria have the option to 
remain in a foster care placement until the age of 2 1 . 

3. The frequency of face-to-face meetings with a case manager was increased from twice 
quarterly to monthly. 

The requirements listed above are more restrictive than those of most other states with federal- 
ly approved extended foster care programs. The vast majority of those states will also allow 
young people to remain in care past age 18 if they are employed for at least 80 hours per 
month or participating in an employment program. 1 Only about 20 percent of the Transitional 
Living Evaluation study sample were receiving either Post-Custody or EFC services at the 
time they enrolled in the study. 

Tennessee Department of Children’s Services (2002). 

Unpublished data from the U.S. Department of Health and Human Services, Administration for 
Children and Families, Children’s Bureau (2016). 
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other services to reduce criminal recidivism. In addition to these federally funded services, 
many states, cities, and counties offer “aftercare” and reentry services for young people who are 
exiting juvenile facilities. 

Research Evidence 

While independent living services have expanded, very few programs have been shown 
by rigorous evaluation to be effective in improving young people’s outcomes. Only four 
moderately sized random assignment evaluations have tested independent living programs for 
young people with a history of foster care (not including the present evaluation), and, among 
those, three did not find any statistically significant impacts — that is, impacts that are larger 
than would generally be expected if the program had no true effect. 10 Rigorous evaluations of 
programs for young people involved in the juvenile justice system have been more common. 
Cognitive behavioral therapy programs, in particular, are supported by a fairly strong research 
base, which has found these programs to be effective in reducing criminal recidivism and 
substance abuse. 11 However, previous studies of programs for juvenile justice-involved young 
people have focused little on measuring program impacts on other important outcomes, such as 
employment, education, and housing. 


The Transitional Living Program 

The Transitional Living program is operated by Youth Villages, a nonprofit social ser- 
vice organization based in Memphis, Tennessee. The organization operates a variety of residen- 
tial and community-based programs serving more than 20,000 young people each year in 12 
states and the District of Columbia. Staff members adhere to a common set of core principles 
and use a common treatment manual, developed by Youth Villages, that contains all of the 
practices the organization considers to be informed by evidence and acceptable for use in its 
programs. 

In Transitional Living, a nonresidential program, services are expected to last an aver- 
age of nine months for individuals who successfully complete the program. Transitional Living 
starts with assessments and the development of an individualized treatment plan that takes into 
account the particular needs and goals of each young person. The bulk of the services are then 
provided during once-a-week, hour-long Transitional Living sessions with a case manager, 
called a “TL Specialist.” 12 Transitional Living sessions are usually held in young people’s 


10 Office ofPlanning, Research and Evaluation (n.d.). 

n Lipsey, Landenberger, and Wilson (2007); Botvin, Baker, Filazzola, and Botvin (1990). 

12 For more detailed information about TL Specialists and the credentials required for this position, see 
Manno, Jacobs, Alson, and Skemer (2014). 
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homes or in the community, often at schools or at fast food restaurants or coffee shop chains. 
Each TL Specialist typically serves only eight young people at a time. 

The topics covered and the activities that take place during Transitional Living sessions 
vary depending on the needs and goals of each participant, but TL Specialists are expected to 
use methods included in the treatment manual. These methods fall into three categories: 
evidence-infomied tools, counseling, and action-oriented activities. Evidence-informed tools 
include specific curricula, such as “Preparing Adolescents for Young Adulthood,” which cover 
topics like money management and job-seeking skills, as well as practices such as the “Adoles- 
cent Community Reinforcement Approach,” which is a behavioral treatment for alcohol and 
other substance abuse. A second strategy involves counseling, in which the participant and TL 
Specialist talk about particular issues in the participant’s life from both the past and the present 
in order to address problems that may be impeding the participant’s progress toward stated 
goals. Finally, TL Specialists use action-oriented activities, such as taking a participant to a 
bank to open an account or to a community college to gather information about classes. 

Aside from direct support that the TL Specialist provides during the regular sessions 
with each young person, Transitional Living offers other resources to participants. All are 
screened for trauma. Trauma-focused cognitive behavioral therapy, a 12- to 20-week course of 
therapy provided by specially trained Youth Villages’ staff, is provided to those for whom it is 
clinically indicated. TL Specialists may also refer participants to other services in the communi- 
ty, such as General Educational Development (GED) classes, specialized mental health ser- 
vices, or housing services. In addition, TL Specialists have access to some flexible funds to 
support those who need money for expenses such as purchasing appropriate clothing for 
interviews or an apartment application fee. They also encourage young people to participate in 
monthly group social and learning activities with others in the program. Youth Villages is 
required to organize these group activities as part of its contract with the Tennessee Department 
of Children’s Services. Finally, educationaFvocational coordinators are available to provide 
extra support to young people who want to go to college, take vocational training, or find a job. 

The set of boxes in the center of Figure 2, which depicts Transitional Living’s logic 
model, shows the domains that are most commonly addressed — education; employment and 
earnings; housing stability and economic well-being; social support; health and safety; and 
criminal involvement — during the sessions with the TL Specialist and through other program 
services. For all domains, services include key program components such as goal planning and 
Transitional Living sessions. Other services, such as the assistance of the educational/vocational 
coordinator, focus on particular domains. Consistent with the individualized nature of the 
Transitional Living program, the extent to which each participant receives services related to 
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Figure 2 

Logic Model for the Youth Villages Transitional Living Program 


Domain-Specific Services 


Transitional Living Services: 
Key Components 

Assessments, individualized 
treatment planning, and goal 
development 

Sessions with TL Specialists, 
including 

o explicit use of evidence- 
informed tools 

o counseling or conversation- 
based interventions 
o action-oriented activities 
Referrals to other services in the 
community 
Financial assistance 
Group social/learning activities 
Educational/vocational 
coordination 



Impacts 












each domain depends on the needs and goals of that individual. The boxes at the far right of 
Figure 2 show the expected effects, or impacts, of Transitional Living services in each of these 
domains. 


The Youth Villages Transitional Living Evaluation 

The Youth Villages Transitional Living Evaluation assessed the impacts of the Transitional 
Living program. Although the program operated in six states during the study period, the 
evaluation tested only the program operating in Tennessee. During the evaluation period, the 
Tennessee program was funded partly by Youth Villages’ contract with the Department of 
Children’s Services and partly by philanthropic support. The study sample includes men and 
women ages 18 to 24 who were living in Tennessee and who had left foster care or juvenile 
justice custody as teenagers or were aging out at 18. The evaluation employed a rigorous 
random assignment design. Between October 2010 and October 2012, 1,322 young people were 
assigned at random to one of two groups: 

• The program group, whose members were offered Transitional Living pro- 
gram services, including intensive case management, support, and counsel- 
ing; 60 percent of the participants in the study were assigned to this group. 

• The control group, whose members were not offered Transitional Living 
program services, but were provided with a list of other social service re- 
sources that were available in the community; 40 percent of the participants 
in the study were assigned to this group. 

While the program group could access other services in the community if they wished, they 
were not provided with the list of resources that was given to the control group. By measuring 
outcomes for both the program and control groups over time, the research team can assess 
whether Transitional Living services led to better outcomes for the program group than for the 
control group. Owing to the random assignment design, the research groups were expected to be 
comparable on both measured and unmeasured characteristics when the study began. Therefore, 
statistically significant differences in outcomes that emerge between the two groups can be 
attributed with some confidence to the offer of Transitional Living services to the program 
group. These differences in outcomes are considered “impacts” or “effects” of the Transitional 
Living program. 

Table 1 presents selected, self-reported background characteristics of the study sample 
at the time of study enrollment. As expected given the random assignment design, there are very 
few statistically significant differences between the program and control groups for these 
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Table 1 


Characteristics of Sample Members at Baseline 

Characteristic (%) Total 


Age categories 

18 years old 71.4 

19 years old 19.4 

20-24 years old 9.2 

Gender 

Male 52.0 

Female 48.0 

Race/ethnicity 

Hispanic 2 5.8 

White, non-Hispanic 51.1 

Black, non-Hispanic 37.1 

Other, non-Hispanic 6.0 

Where participant lived 

Own apartment/house 5.5 

Home of biological parent(s) 28.7 

Home of other relative(s) 19.4 

Home of friend(s) 10.9 

Foster home 20.9 

Homeless/living on the street 1.7 

Supervised independent living arrangement 1.8 

Group home, halfway house, or residential treatment center 4.4 

Other 6.7 

Ever employed 54. 1 

Employed at baseline 19.2 

Educational attainment and school enrollment 
No high school diploma or GED certificate, not enrolled in school 17.3 

No high school diploma or GED certificate, enrolled in school 40.0 

High school diploma or GED certificate, 
not enrolled in postsecondary school 29.1 

High school diploma or GED certificate, 

enrolled in postsecondary school 13.7 

Ever repeated a grade or been held back a grade 43.3 


(continued) 
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Table 1 (continued) 


Characteristic (%) Total 

Ever been suspended from school 80.5 

Ever been in special education 25.6 

Contact with biological mother 

Everyday 43.1 

At least once a week but not every day 16.4 

At least once a month but not every week 7.7 

Less than once a month 8. 1 

Never 24.7 

Contact with biological father 

Everyday 16.4 

At least once a week but not every day 13.1 

At least once a month but not every week 7.7 

Less than once a month 10.3 

Never 52.5 

Had contact with any other relatives at least once per month 88.4 

Pregnant at baseline 4. 1 

Had any children 16.9 

Ever arrested 64.4 

Received psychological or emotional counseling in past year 55.7 

Attended substance abuse treatment program in past year 31.1 

Sample size 1,322 

SOURCE: MDRC calculations based on responses to the Youth Villages 
Transitional Living Evaluation Baseline Information Form. 

NOTES: GED = General Educational Development. 

Rounding may cause slight discrepancies in calculating sums and differences. 
“Sample members are coded as Hispanic if they answered "yes" to Hispanic 
ethnicity. 
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characteristics. Therefore, for simplicity, the table presents numbers for the full study sample. 13 
Similar to other young people with histories of foster care or juvenile justice custody, those who 
enrolled in the study averaged relatively low levels of educational attainment and employment 
at study entry, while experiencing relatively high rates of arrest and housing instability (as 
indicated by the proportion still living in a foster home, in the home of a friend, in a supervised 
independent living arrangement, in a group home, a halfway house, a residential treatment 
center, or living on the street). The young people in the study are diverse in terms of gender and 
race: 48 percent of the sample are women, over 50 percent are white/non-Hispanic, and a 
substantial minority are black/non-Hispanic (37 percent). 

As shown in Table 2, which provides self-reported information on the state custody his- 
tories of sample members, the young people enrolled in the study come from varied custody 
backgrounds, with their first custody placement — often one of many — tending to occur in 
their teens. Sixty-one percent of the sample reported having been in custody because they had 
been neglected or abused (foster care), while 52 percent indicated that they had been in custody 
for delinquency (juvenile justice). About 13 percent of the study sample had experienced both 
types of custody (not shown in table). 


Program Implementation and Service Receipt Differences 
Between the Two Research Groups 

To help interpret the impacts of the Transitional Living program, the research team studied the 
strength of the program’s implementation and the dosage (level and intensity) of services that 
program group members received. Transitional Living was implemented largely in accordance 
with the program model. Although the Transitional Living program had considerable structure, 
the TL Specialists had a great deal of flexibility to adapt services based on the specific needs of 
the individuals in their caseloads, including employment, housing, education, life skills, and 
mental health. TL Specialists were instructed to choose the strategies they used in Transitional 
Living sessions to capitalize on the strengths of each participant. In general, strategies fell 
within the three broad categories (discussed above) that TL Specialists were expected to use: 
evidence-infonned tools, counseling, and action-oriented activities. 


13 See Valentine, Skemer, and Courtney (2015) for a more detailed comparison of the characteristics of the 
two groups. 
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Table 2 


State Custody History of Sample Members at Baseline 


Characteristic (%) 

Total 

Ever in state custody because of 

Neglect, abuse, or unruly adjudication (foster care) a 

61.3 

Delinquency (juvenile justice) 

51.9 

Age in years at first custody entry 

0-5 

6.5 

6-10 

6.3 

11-14 

23.2 

15-16 

32.3 

17-18 

31.7 

Age in years at final custody exit 

1 6 or under 

4.8 

17 

27.6 

18 or over 

39.4 

Still in custody at baseline 

28.3 

Number of different custody placements 

1 placement 

34.8 

2-5 placements 

49.8 

6-10 placements 

9.8 

More than 10 placements 

5.6 

Sample size 

1,322 


SOURCE: MDRC calculations based on responses to the Youth Villages 
Transitional Living Evaluation Baseline Information Form. 

NOTES: Rounding may cause slight discrepancies in calculating sums and 
differences. 

a An unruly adjudication occurs because children are determined to have 
behavioral problems serious enough that their health and safety are at risk or 
because they have committed an offense, such as truancy, that is applicable 
only to minors. 


A substantial portion of the program group received services at the expected average 
dosage of the Transitional Living program model. About two-thirds participated in program 
services for at least five months and about half participated for at least nine months, the ex- 
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pected average length of services for those who successfully complete the program. (Partici- 
pants who achieved their treatment goals early in their participation could be discharged from 
the program before reaching a particular length of stay.) Nearly all program group members 
participated in at least one program activity; 95 percent participated in at least one Transitional 
Living session. While involved in the program, each individual participated in nearly one 
session per week, averaging over an hour per session. Data measuring program participation for 
up to 12 months following random assignment show that, in total, program group members 
averaged about 26 sessions with their TL Specialist. 14 During these sessions, TL Specialists and 
participants covered a wide range of issues, with education, employment, and housing discussed 
most often. 

While control group members could not receive Transitional Living services, they were 
able to obtain other services available in the community, including extended foster care services 
provided by the state to those who were eligible. Therefore, using data from the survey adminis- 
tered one year after random assignment, the research team assessed the extent to which the offer 
of the Transitional Living program increased the services received by the program group over 
what the control group received. 

Overall, there are large, statistically significant differences between the program and 
control groups in the dosage of the services they received. The program group was more likely 
than the control group to have had a case manager or social worker (75 percent compared with 
44 percent), who could be a TL Specialist, 15 and to have met with that person at least once a 
week (60 percent compared with 20 percent). They were more likely to have received help, 
from any source, with problems related to education, employment, finances, housing, and daily 
living. However, while there was a clear difference in the level of services received, it is notable 
that many control group members also obtained case management and other services. 


One-Year Impacts of the Transitional Living Program 

The primary source of outcome data for the one-year impact report is a survey that was admin- 
istered by NORC at the University of Chicago, largely by telephone, to all sample members 
about one year after they entered the study. The response rate was 84.3 percent. Outcomes in six 
key domains were covered: education; employment and earnings; housing stability and eco- 

14 About 19 percent of program group members participated in Transitional Living for more than 12 
months. TL sessions that occurred more than 12 months after random assignment are not included in this 
average. 

15 While it is likely that most program group members were referring to TL Specialists when reporting help 
they received from a case manager or social worker, some may have had case managers or social workers from 
outside of the Transitional Living program, including case managers from the Tennessee Department of 
Children’s Services. 
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nomic well-being; social support; health and safety; and criminal involvement. In addition, the 
research team collected administrative data on postsecondary enrollment from the National 
Student Clearinghouse. Before conducting the one-year impact analysis, the research team 
specified primary outcomes and secondary outcomes within each of the six domains. Conclu- 
sions about the effectiveness of the Transitional Living program in each domain depend on the 
impact estimates for primary outcomes. (For an overall explanation of the differences between 
primary and secondary outcomes, along with guidance on how to correctly interpret findings for 
these two groupings, see Box 2.) 

Statistically significant impacts on primary outcomes were detected in three domains: 
employment and earnings; housing stability and economic well-being; and health and safety. In 
the area of employment and earnings, the program led to an increase of over $600 in earnings in 
the year before the survey interview, driven, at least in part, by an increase in the percentage of 
young people who were employed. In the area of housing stability and economic well-being, the 
research team found that program group members experienced significantly fewer types of 
housing instability than control group members, most notably reductions in homelessness and 
couch surfing (staying in someone else’s home temporarily when not having a permanent place 
to live). Similarly, the Transitional Living program reduced incidence of economic hardship, 
driven by decreases in the percentage of those who did not have necessary clothing or shoes and 
in the percentage of young people who had delayed paying a bill in order to buy food. Finally, 
in the health and safety domain, Transitional Living improved mental health and reduced the 
percentage of those involved in violent relationships. However, it did not significantly affect 
other key measures of health and safety, including substance use, condom use, and victimiza- 
tion. No statistically significant effects were found on primary outcomes in the remaining three 
domains: education, social support, and criminal involvement. 

While the one-year analysis focused primarily on impacts for the full research sample, 
there were reasons to hypothesize that the Transitional Living program may have larger or 
smaller impacts for some subgroups of young people compared with others. Accordingly, the 
research team assessed differences in impacts across four sets of subgroup characteristics, 
described later in this report. In the one-year analysis, the research team found that the impacts 
of Transitional Living were consistent across these subgroups, indicating that the program is 
equally effective across the subgroups analyzed. 

Overall, while the statistically significant impacts detected at one year proved modest, 
their breadth across several domains is consistent with the highly individualized nature of the 
program model, which is designed to address the wide variety of needs and circumstances of the 
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Box 2 

Approach to the Impact Analysis in This Report 


The Transitional Living program provides individualized services to young people who have a 
wide variety of experiences, needs, and circumstances. The program aims to improve out- 
comes across multiple domains rather than focusing on one or two outcomes as some pro- 
grams do. Therefore, the Youth Villages Transitional Living Evaluation research team esti- 
mated the two-year impacts of the Transitional Living program on outcomes in three distinct 
domain s : education; employment and earnings; and criminal involvement. 

While this approach makes sense theoretically, examining a large number of outcomes in- 
creases the chance of observing a significant impact even if the program has had no true effect 
— that is, an impact that arises by chance alone. A statistically significant impact estimate is 
one that is unlikely to have occurred if the program was truly ineffective. When an impact 
estimate is statistically significant at the 10 percent level, for example, it means that there is 
only a 10 percent chance that an ineffective program would have generated this estimate. 
Increasing the number of impact estimates examined further increases the chance that a 
significant impact will be found for an ineffective program. For example, if 10 independent 
outcomes are examined, there is a 65 percent chance that one of them will be statistically 
significant at the 10 percent level purely by chance, even if the program is truly ineffective for 
that outcome. 

To guard against the possibility of drawing wrong conclusions about the effectiveness of the 
Transitional Living program, the research team prespecified a small number of primary 
outcomes within each of the three outcome domains. The primary outcomes selected for this 
analysis are broader, more comprehensive measures. Conclusions about the effectiveness of 
the Transitional Living program in a particular domain hinge on the impact estimates for 
primary outcomes. To provide additional detail about where impacts on primary outcomes 
were concentrated, the team also prespecified secondaiy outcomes in each domain. These 
estimates do not shape conclusions about the effectiveness of the Transitional Living program 
in particular domains but, rather, flesh out the story where there are impacts. The discussion of 
impact results in this report concentrates on primary outcomes. 


young people it serves. Taken together, the one-year impact findings were promising, especially 
given the lack of statistically significant, positive impacts for other programs targeting similar 
populations. 

A more detailed description of the Transitional Living program model and discussion of 
its implementation can be found in the first report in this evaluation, Moving Into Adulthood . 16 
Similarly, a more in-depth discussion of service receipt differences between program and 


16 Manno, Jacobs, Alson, and Skemer (2014). 
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control group members and estimated one-year impacts of the Transitional Living program are 
available in the second report in this evaluation, Becoming Adults . 17 


17 Valentine, Skemer, and Courtney (2015). 
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Two-Year Impacts on Education, Employment and 
Earnings, and Criminal Involvement 


The one-year Transitional Living impact results were promising, particularly in certain do- 
mains, but they represent only short-term impacts of the program. Examining longer-term 
impacts is important, as Transitional Living is designed not only to support young people while 
they are in the program, but also to provide them with the tools to better navigate adult life after 
they leave the program. Many participants were involved in Transitional Living throughout the 
one-year follow-up period; it is important to know whether the impacts found at one year were 
sustained into a second year, after the vast majority of participants had exited the program. 

To this end, the two-year impact analysis capitalizes on the existence of various admin- 
istrative data sets to assess whether some of the statistically significant impacts found after one 
year were sustained into a second year and whether new impacts emerged over time. The two- 
year analysis estimates impacts in three domains: (1) education, using postsecondary enrollment 
data from the National Student Clearinghouse; (2) employment and earnings, using unemploy- 
ment insurance data from the Tennessee Department of Labor and Workforce Development; 
and (3) criminal involvement, using arrest and conviction data from the Tennessee Bureau of 
Investigation. For all three administrative data sources, the research team created Year 1, Year 
2, and overall outcome measures and estimated impacts on these measures. With the exception 
of postsecondary enrollment data, Year 1 impacts on outcomes measured using administrative 
data were not assessed or presented in the one-year report. 

Given that the study does not include a second follow-up survey and that administrative 
data are available only for certain outcomes, the research team is not assessing two-year impacts 
in three of the original six outcome domains: housing stability and economic well-being; social 
support; and health and safety. 18 As a result, the research team is not able to assess whether 
impacts were sustained into a second year in two of the domains — housing stability and 
economic well-being and health and safety — in which there were statistically significant 
effects at one year. Therefore, while two-year impacts in the domains for which data are 
available remain informative, this analysis presents only a partial assessment of the effective- 
ness of the Transitional Living program after two years of follow-up. 


18 The original design for the study did not include a two-year impact analysis or the fielding of a longer- 
term survey, as the evaluation team decided to assess whether the program produced short-term impacts before 
collecting additional follow-up data. Given the timing of the one-year impact analysis, a second follow-up 
survey would not have been fielded until at least four years after random assignment, by which time it would 
have been very difficult and extremely costly to obtain high response rates among the study sample. Therefore, 
the research team decided to pursue administrative data only. 
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Impacts on Education 

Educational attainment is of great importance for young people as they attempt to make a 
successful transition to adulthood. 19 Increasingly, those with low levels of education struggle to 
gain a foothold in the labor market and are far more likely than their peers with higher levels of 
education to live below the poverty line. 20 For those who have spent time in the foster care or 
juvenile justice systems, educational attainment is no less important. However, on average, 
young people with custody histories face a more difficult path forward, as many have fallen 
well behind their peers on key educational indicators. 21 In an effort to bridge this gap, TL 
Specialists strongly emphasize education by working with participants to establish educational 
goals and actively supporting them in taking the various steps necessary to achieve their goals. 

As discussed in the introduction to this report, the one-year impact analysis shows no 
statistically significant impacts of the Transitional Living program on education. Educational 
outcomes measured at one year include high school diploma receipt, GED receipt, participation 
in vocational training, and postsecondary enrollment. However, a large percentage of sample 
members were still working toward completing high school at baseline, and some who aspired 
to college may have needed a bit more time to realize this goal in light of the many challenges 
they faced. Therefore, the research team examined whether statistically significant differences 
in postsecondary enrollment emerged between the program and control groups over a second 
year of follow-up. 

Table 3 presents two-year impacts of the Transitional Living program on education. 
(For a detailed explanation of how to read the impact tables in this report, see Box 3.) The 
primary outcome in this domain is enrollment in a postsecondary institution. Transitional Living 
did not produce a statistically significant impact on this outcome, either across the two years of 
follow-up or looking at each year individually. About a quarter of both program and control 
group members enrolled in a postsecondary institution at some point in the two years following 
study enrollment. For both research groups, postsecondary enrollment declined in Year 2 from 
the levels observed in Year 1. 


19 Child Trends (2014). 

20 Caspi, Wright, Moffitt, and Silva (1998); U.S. Department of Education (2012); Aud, KewalRamani, 
and Frohlich (2011). 

21 Courtney, Dworsky, Lee, and Raap (2010); Leone and Weinberg (2010). 
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Table 3 


Two-Year Impacts on Education 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P- Value 

Primary outcomes 

Enrolled in a postsecondary institution 

22.0 

25.4 

-3.4 

0.111 

Year 1 

18.9 

18.4 

0.4 

0.821 

Year 2 

11.4 

13.6 

-2.2 

0.212 

Secondary outcomes 

Type of postsecondary institution (Years 1-2) 

Enrolled in 4-year college 

8.1 

10.3 

-2.2 

0.154 

Enrolled in 2-year college 

15.8 

16.8 

-1.1 

0.595 

Sample size (total = 1,322) 

788 

534 




SOURCE: MDRC calculations based on postsecondary education data from the National Student 
Clearinghouse. 


NOTES: Results in this table are regression-adjusted, controlling for pre -random assignment 
characteristics. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 


Although there are no statistically significant impacts on primary education outcomes, 
an analysis of secondary measures sheds additional light on the outcomes of sample members. 
Again, there are no statistically significant differences between the program and control groups 
in their rates of enrollment in either four-year or two-year colleges. Eight percent of program 
group members and 10 percent of control group members enrolled in a four-year college. 
Members of the study sample more commonly attended two-year colleges: 16 percent of 
program group members and 17 percent of control group members enrolled in a two-year 
college during the two-year follow-up period. 
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Box 3 

How to Read the Impact Tables in This Report 

The impact tables in this report use a similar format, illustrated below. In this case, employ- 
ment and earnings outcomes are shown for the program group and the control group. For 
example, the table shows that 36.6 percent of the program group earned $2,500 or more in 
Year 2, while 30.3 percent of the control group earned over this same threshold. 

The “Difference” column in the table below shows the differences between the two research 
groups’ proportion earning $2,500 or more in Year 2 — that is, the program’s estimated effect, 
or impact, on earnings at this level. For example, the estimated impact on this outcome can be 
calculated by subtracting 30.3 from 36.6, yielding a 6.3 percentage point difference. 

Differences marked with asterisks are “statistically significant,” meaning that they are larger 
than would generally be expected if the program had no true effect; thus, they are likely 
attributable to the offer of the program services. The number of asterisks indicates whether the 
estimated impact is statistically significant at the 10 percent (one asterisk), 5 percent (two 
asterisks), or 1 percent (three asterisks) level — the lower the level (or the more asterisks), the 
less likely that the impact could have been generated by an ineffective program. For example, 
as shown in the first row of data, the Transitional Living program had a statistically significant 
impact of 6.3 percentage points on the proportion of young adults earning over $2,500 during 
Year 2; that is, recipients of Transitional Living services were 6.3 percentage points more 
likely, on average, to earn above this threshold than those who had not been offered Transi- 
tional Living services. This impact is statistically significant at the 5 percent level — meaning 
that there is less than a 5 percent probability that an ineffective program would have resulted in 
an estimated impact this large. The p-value shows the exact level of significance. 


Two-Year Impacts on Employment and Earnings 


Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Primary outcomes 





Y ear 2 earnings (%) 

$2,500 or more 

36.6 

30.3 

6.3 ** 

0.016 

$5,000 or more 

22.9 

19.2 

3.7 

0.105 

$7,500 or more 

14.2 

13.1 

1.1 

0.566 

Sample size (total = 1,322) 

788 

534 
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Impacts on Employment and Earnings 

Employment is another key area in which young people with histories of foster care or juvenile 
justice custody lag behind their peers; they experience comparatively high rates of unemploy- 
ment and low earnings. 22 The young people in the study sample were not well-positioned at 
baseline to disrupt these trends; just 19 percent of study participants reported holding a job at 
the time they enrolled in the study, about half the rate of young people of a similar age in the 
general population. 23 

As noted earlier, the one-year analysis, based on self-reported data collected in survey 
interviews, showed that Transitional Living was able to significantly improve employment and 
earnings. Program group members earned over $600 more than control group members from 
formal employment in the year before their survey interview and were 5 percentage points more 
likely to be employed, both statistically significant effects. To determine whether these positive, 
statistically significant impacts persisted after the one-year follow-up period, the research team 
collected state unemployment insurance data from Tennessee to estimate employment and 
earnings impacts at two years after study enrollment. 

Table 4 presents two-year impacts of the Transitional Living program on employment 
and earnings based on administrative data. The primary outcome in this domain is earnings. To 
best understand the effect of the Transitional Living program on this outcome, the research team 
examined earnings in multiple ways. First, the team estimated impacts on average earnings in 
Year 1, Year 2, and across both years of follow-up. No statistically significant differences 
between program and control group members were observed for any of these measures when 
administrative data were used. Total earnings for both research groups hovered at about $5,000, 
with earnings increasing from approximately $2,000 in Year 1 to approximately $3,000 in Year 
2. Overall, these earnings levels are quite low. As indicated by the employment rates presented 
in the bottom panel of Table 4 and discussed further below, many young people in the study 
sample had no formal employment and hence no formal earnings, thus drawing down the 
average earnings estimates. 

The research team also assessed impacts on earnings using what are referred to as 
“binned” variables, or variables that capture the percentage of sample members falling into 
certain categories. In this instance, the team analyzed the percentage of each research group 
earning more than a particular threshold in each year. These thresholds were $2,500 or more, 
$5,000 or more, and $7,500 or more. The thresholds are not mutually exclusive: for example, 


22 Courtney et al. (2010); U.S. Department of Health and Human Services (2008); Sampson and Laub 
(1990); Ramchand, Morral, and Becker (2009). 

23 See U.S. Department of Labor, Bureau of Labor Statistics (2014). 
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Table 4 


Two-Year Impacts on Employment and Earnings 


Outcome 3 

Program 

Group 

Control 

Group 

Difference 

(Impact) 


P-Value 

Primary outcomes 

Total earnings ($) 

5,240 

5,016 

224 


0.555 

Year 1 

2,233 

2,130 

103 


0.562 

Year 2 

3,006 

2,885 

121 


0.641 

Y ear 1 earnings (%) 

$2,500 or more 

30.5 

25.5 

5.1 

** 

0.035 

$5,000 or more 

16.7 

13.4 

3.3 

* 

0.083 

$7,500 or more 

9.0 

7.9 

1.1 


0.462 

Y ear 2 earnings (%) 

$2,500 or more 

36.6 

30.3 

6.3 

** 

0.016 

$5,000 or more 

22.9 

19.2 

3.7 


0.105 

$7,500 or more 

14.2 

13.1 

1.1 


0.566 

Secondary outcomes 

Ever employed (%) 

78.5 

75.6 

2.9 


0.212 

Year 1 

64.1 

59.3 

4.8 

* 

0.072 

Year 2 

63.6 

60.0 

3.5 


0.195 

Number of quarters employed 

3.2 

3.0 

0.2 

* 

0.088 

Sample size (total = 1,322) 

788 

534 





SOURCE: MDRC calculations based on unemployment insurance quarterly earnings data from the 
Tennessee Department of Labor and Workforce Development. 


NOTES: Results in this table are regression-adjusted, controlling for pre-random assignment 
characteristics. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

““Unemployment insurance data are quarter-based; thus, the two-year follow-up period in this table 
does not necessarily represent the 24 months following random assignment. Rather, the follow-up 
period represents the eight quarters following the quarter in which participants were randomly assigned. 
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someone earning $10,000 per year would be included in each bin. The benefit of this approach 
is that binned variables are not sensitive to the effects of high (but still plausible) values, which 
may have an undue upward effect on averages. Relatedly, this approach allows for an improved 
understanding of how impacts may vary at different earnings levels, since averages may not tell 
the whole story. 

The Transitional Living program had a statistically significant, 5-percentage-point im- 
pact on the proportion of young people earning $2,500 or more in Year 1, with 31 percent of the 
program group and 26 percent of the control group exceeding this threshold. The program also 
had a statistically significant effect on the proportion of young people earning $5,000 or more in 
Year 1 (17 percent of program group members versus 13 percent of control group members). 
The program’s significant effect diminished in the highest earnings “bin”: There are no statisti- 
cally significant differences between the proportion of program and control group members 
earning $7,500 or more in Year 1 . 

Impacts on binned earnings variables in Year 2 follow a somewhat similar pattern to 
those observed in Year 1. There is a statistically significant, 6-percentage-point impact on the 
proportion of young people earning $2,500 or more in Year 2 (37 percent of program group 
members versus 30 percent of control group members). 24 The program’s significant effect 
diminishes as earnings thresholds increase, with no statistically significant differences between 
program and control group members earning either $5,000 or more or $7,500 or more in Year 2. 

There are two secondary outcomes in this domain, employment and number of quarters 
employed. Transitional Living did not produce a statistically significant impact on the percent- 
age of young people who were ever employed during the two-year follow-up period; over three- 
fourths of both the program group and the control group had worked in jobs covered by unem- 
ployment insurance. 25 However, consistent with the one-year impact findings based on survey 
data, the program group was about 5 percentage points more likely than the control group to 
have been employed in Year 1, a statistically significant difference that had faded by Year 2. 
Program group members appear to have been somewhat more consistently employed over the 
two-year follow-up period, working an average of 3.2 quarters compared with 3.0 among 
control group members. 


24 As a result of rounding, the difference between the program and control group proportions earning over 
$2,500 in Year 2 appears to calculate to 7 percentage points. However, the difference between unrounded 
means results in a differential of 6.3 percentage points. 

^Administrative data are collected only for jobs covered by unemployment insurance. For this reason, the 
two-year analysis of impacts on employment and earnings includes only these jobs. 
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Comparing Survey Data with Administrative Data 

While employment figures match up fairly well between self-reported one-year survey 
data and one-year administrative data, some discrepancies are seen between these two data 
sources for earnings during the first year of follow-up. These discrepancies arise at both the 
levels (one-year earnings based on survey data are higher for both research groups than one- 
year earnings based on administrative data) and the estimated impacts (the one-year impact on 
earnings based on survey data is about $600 and is statistically significant, while the one-year 
impact on earnings based on administrative data is about $100 and is not statistically signifi- 
cant). 


A few important differences between survey data and administrative data may shed 
some light on these discrepancies. The survey, though based on self-reporting and therefore 
subject to participants’ recollections, may have captured some jobs that are not included in 
administrative data from the Tennessee unemployment insurance system. First, while study 
participants were instructed to report only formal employment in answer to the questions used 
to create the survey-based one-year outcomes, they may very well have reported work in some 
jobs in the informal economy, such as babysitting or mowing lawns. Second, they may also 
have reported formal employment where the worker is classified as an independent contractor, 
such as an Uber driver. Neither of these types of employment are captured by unemployment 
insurance data. (Data from the unemployment insurance system also do not capture employ- 
ment by some categories of workers, such as those who are self-employed, federal government 
employees, and domestic workers.) Finally, about 8 percent of survey respondents reported 
working outside of Tennessee; this employment, as well as the earnings resulting from it, would 
not have been picked up in the Tennessee-provided administrative data. 

In addition, the survey sample includes 1,114 of the 1,322 young people in the full 
study sample, while administrative data capture information for the full research sample of 
1,322. Survey response bias analysis suggests that this last difference has little effect on impact 
estimates, but it may partly explain the lower earnings levels observed across both research 
groups in the administrative data relative to the survey data: Survey respondents tend to be those 
who are faring better and are therefore likely to be more consistently employed and earning 
more than nonrespondents. However, because this is true of respondents in both research 
groups, impact estimates should not be biased. 

Informal or contract employment not covered by unemployment insurance data and 
employment outside of Tennessee may contribute to the discrepancies between survey-based 
earnings and administrative data-based earnings in terms of both differing earnings levels and 
differing impacts. The administrative data are missing some amount of earnings for both 
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research groups by not capturing infonnal employment, contract employment, and employment 
outside of Tennessee. 26 However, the differences in impact estimates between the two sources 
suggest that program group members may be differentially affected by these gaps in the 
administrative data, indicating that Transitional Living may have increased infonnal employ- 
ment, contract employment, and/or employment outside of the state of Tennessee. 

Given the pros and cons of both survey and administrative data in measuring employ- 
ment and earnings outcomes, it is useful to have access to both sources in order to obtain the 
fullest possible understanding of the effects of the Transitional Living program in this domain. 


Impacts on Criminal Involvement 

While most young people with a history of delinquent behavior do not go on to commit crimes 
as adults, they are at higher risk of committing adult crimes relative to those without a history of 
delinquency. 27 Almost two-thirds of the young people in the Transitional Living study sample 
had been arrested before random assignment occurred; about half had exhibited delinquent 
behavior that was serious enough to result in juvenile justice custody, indicating that many in 
the sample were at high risk of criminal involvement as adults. 

The one-year impact results, based on survey data, showed that the Transitional Living 
program did not significantly reduce criminal involvement. However, given the importance of 
the outcomes in this domain to policymakers and the high societal costs attached to contact with 
the criminal justice system, administrative data were gathered to analyze whether any changes 
to the impact results would emerge with longer-term follow-up. 

Table 5 presents two-year impacts of the Transitional Living program on criminal in- 
volvement, or, more specifically, contact with the criminal justice system. There are two 
primary outcomes in this domain: arrest and conviction. There are no statistically significant 
differences between program and control group members in their rates of arrest. Just under half 
of the members of both groups were arrested at some point in the two years following study 
enrollment. About one-third of sample members were arrested in Year 1, with the same being 
true of Year 2. Similarly, Transitional Living did not produce a statistically significant impact 
on conviction. About one- fifth of both research groups were convicted of a crime during the 
two years following study enrollment. 


Because Tennessee borders seven other states, out-of-state employment may be relatively prevalent 
among the study sample. 

27 Piquero, Hawkins, and Kazemian (2012). 
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Table 5 


Two-Year Impacts on Criminal Involvement 


Outcome (%) 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Primary outcomes 

Arrested 

47.7 

47.6 

0.1 

0.972 

Year 1 

30.7 

31.1 

-0.4 

0.877 

Year 2 

34.0 

34.9 

-0.9 

0.721 

Convicted of a crime 

19.7 

17.7 

2.0 

0.350 

Year 1 

10.5 

9.5 

1.0 

0.564 

Year 2 

13.4 

12.6 

0.9 

0.635 

Secondary outcomes 

Conviction class (Years 1-2) 

Convicted of a felony 

6.4 

5.3 

1.1 

0.398 

Convicted of a misdemeanor 

15.6 

15.3 

0.4 

0.861 

Conviction categories (Years 1-2) 

Convicted of a violent crime 

4.4 

3.0 

1.4 

0.189 

Convicted of a property crime 

7.7 

5.7 

2.0 

0.167 

Convicted of a drug crime 

4.5 

4.7 

-0.2 

0.858 

Convicted of a public order crime 

9.8 

9.9 

-0.1 

0.966 

Sample size (total = 1,322) 

788 

534 




SOURCE: MDRC calculations based on criminal history data from the Tennessee Bureau of 
Investigation. 


NOTES: Results in this table are regression-adjusted, controlling for pre-random assignment 
characteristics. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 


There are two secondary measures in this domain: conviction class — the percentage of 
those convicted of a felony versus the percentage of those convicted of a misdemeanor, and 
conviction category — the percentage convicted of violent, property, drug, and public order 
crimes. There are no statistically significant differences between program and control group 
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members in their rates of felony or misdemeanor conviction. Six percent of program group 
members and 5 percent of control group members were convicted of a felony, the more serious 
conviction class. Sixteen percent of program group members and 1 5 percent of control group 
members were convicted of a misdemeanor. Transitional Living also appeared to have no 
statistically significant effect on different conviction categories. Comparable, small proportions 
of program and control group members were convicted of violent, property, drug, and public 
order crimes in the two years after they enrolled in the study. Public order crimes, usually 
considered to be less serious offenses, were most common. In general, most of the crimes 
committed by members of the study sample were minor in nature. 


Impacts by Subgroups of Young People 

As discussed earlier, there are reasons to hypothesize that the impacts of the Transitional Living 
program may be different for some subgroups of young people compared with others. Early in 
the evaluation, before conducting any impact analyses, the research team hypothesized that the 
pattern of impacts might differ across four specific sets of subgroup characteristics. The team 
initially conducted subgroup analyses at one year of follow-up. These analyses examined the 
pattern of impacts on primary outcomes by history of juvenile justice custody; by geographic 
setting; 28 by receipt of Extension or Re-Establishment of Foster Care (EFC) Services (formerly 
Post-Custody Services) at baseline; 29 and by latent classes, or clusters identified by an analysis 
meant to group young people based on their readiness for independent living using key baseline 
characteristics. 30 As summarized earlier, none of the four subgroup analyses uncovered a strong 
pattern of significantly different impacts as of one year of follow-up. That is, the results indicat- 
ed that the impacts of Transitional Living were consistent across all of the subgroups. 

For the two-year impact analysis, the research team assessed subgroup impacts for these 
same four prespecified subgroups in order to determine whether the one-year results held true 
with longer-term follow-up. Once again, no strong pattern of significantly different impacts 
emerged; the overall conclusion remains that the impacts of the Transitional Living program are 
consistent across the four subgroups analyzed. All tables related to the subgroup analysis, 
including a table containing baseline characteristics by latent class assignment, are found in 

28 Youth who lived in zip codes where 70 percent or more of the inhabitants were living on urban blocks, 
as defined by the U.S. Bureau of the Census, were considered to be living in urban areas. 

29 The change from Post-Custody Services to EFC Services went into effect in July 2012. For simplicity, 
this discussion refers to all such services as EFC Services, but those that were received before July 2012 were 
Post-Custody Services. 

30 The latent class analysis identified three subgroups of young people: those who were "hindered but con- 
nected to family,” “maltreated but avoiding trouble,” and “long-term system-involved but engaged” (that is, 
involved with the foster care or juvenile justice system over a long period but engaged with employment and 
education). 
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Appendix A of this report. More detailed information about the various subgroups and how they 
were created is available in the second MDRC report on the Youth Villages Transitional Living 
Evaluation, Becoming Adults . 31 


Conclusion 

The research team assessed program impacts in three key domains two years after participants 
enrolled in the study, well after the vast majority of program group members had left the 
Transitional Living program, to leam whether the impacts found at one year were sustained and 
whether new impacts had emerged over time. The Transitional Living program maintained 
some modest effects on employment and earnings outcomes, though evidence of the program’s 
effects in this domain are stronger for Year 1 than Year 2. Notably, earnings levels based on 
administrative data were quite low for both research groups, underscoring the level of disad- 
vantage experienced by young people in the study. 

Statistically significant improvements in education and criminal involvement did not 
emerge with longer-term follow-up for the outcomes that were measured in these domains. The 
lack of significant impacts on education is disappointing, given the program’s emphasis in this 
area. However, many of these young people were likely in dire need of income or may have 
faced more urgent problems related to housing instability or personal safety that precluded them 
from pursuing higher education. Additionally, administrative records from the National Student 
Clearinghouse do not include information about credit attainment, meaning that while the 
research team was able to assess Transitional Living’s impact on postsecondary enrollment, it 
was not able to investigate whether the program had an effect on progress toward a degree. 

Overall, approximately 80 percent of the study sample were either enrolled in postsec- 
ondary education (including vocational training offered through community colleges or partici- 
pating technical schools) or working in an unemployment insurance-covered job at some point 
over the two-year follow-up period, indicating that most of the sample members were involved 
in gainful activity of some type. The true percentage of those who were productively engaged is 
likely to be even higher, as some vocational training (vocational training not reported to the 
National Student Clearinghouse) and pursuit of a high school diploma or GED certificate are 
not included in this measure because of data limitations. 

With regard to criminal involvement, much research has shown that it is very difficult 
to improve outcomes in this domain, even when reducing criminal behavior and contact with 
the criminal justice system is the main focus of a program. Transitional Living is designed to 
address a number of other issues in participants’ lives in addition to criminal involvement. 

31 Valentine, Skemer, and Courtney (2015). 
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Services related to criminal justice issues consisted of case management and counseling from 
TL Specialists, including some interventions related to risky behaviors. However, the program 
did not have additional components that explicitly focused on changing criminal behavior as it 
did for some other domains — one example being the use of educational/vocational coordina- 
tors to assist participants with education- and employment-related pursuits. 

Finally, concluding the summary of two-year impact findings, the research team found 
little evidence that the impacts of the Transitional Living program varied across different 
subgroups of young people. 

Overall, these findings are fairly consistent with those found after the first year of fol- 
low-up. Bear in mind, though, that the research team was unable to assess two-year impacts in 
three domains, including two domains in which positive, statistically significant effects were 
detected based on one-year survey data. Therefore, the results presented in this report do not 
provide a full assessment of the two-year impacts of the Transitional Living program. In 
particular, it is unknown whether statistically significant impacts on housing stability and 
economic well-being and health and safety persisted into a second year. 
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Cost of the Transitional Living Program 


This section presents the estimated cost of providing Transitional Living services to sample 
members. The cost analysis uses data, provided by Youth Villages, on program expenditures 
and staffing schedules covering the period during which most of the program group were 
receiving Transitional Living services. 32 Table 6 presents infonnation on the costs per sample 
member of operating the Transitional Living program. The cost per sample member describes 
the resources required to operate the Transitional Living program for the 788 young people who 
were offered services in Tennessee during the evaluation period. 33 As shown in Table 6, the 
estimated total cost per sample member was $ 1 1 ,84 1 . 34 

Three out of four dollars spent on the Transitional Living program paid for staff salaries 
and benefits ($8,952 per sample member). Nearly 80 percent of this amount paid for the salaries 
and benefits of staff providing direct services to participants (60 percent of all spending), while 
the other approximately 20 percent paid for program administration (16 percent of all spending). 
The second- and third-largest spending categories were corporate overhead and travel, each 
representing nearly 10 percent of all spending. The travel category includes the costs of TL 
Specialists traveling to meet with young people. The remaining spending paid for operations 
and maintenance (3 percent of all spending) and direct support to participants (2 percent of all 
spending). Direct support to participants includes small payments provided to those who 
required financial assistance with basic needs and other living expenses, such as clothing, food, 
tickets for public transit, housing, and utility bills. 

The cost of the program per sample member presented here is a gross estimate rather 
than a net estimate. In other words, because of lack of available infonnation on the costs of 
other services, the net cost of the Transitional Living program, or the cost of services to program 
group members over and above the cost of services to control group members, is not offered 
here as it would be in a fonnal comparison of costs and benefits. (While the research team 
planned to conduct a fonnal benefit-cost analysis as part of this report, the team ultimately 
decided not to move forward for two main reasons. First, the program’s impacts, or benefits, are 


32 Data cover fiscal years 2011 to 2012 and 2012 to 2013. 

33 This approach includes young people who were assigned to the program group but never received Tran- 
sitional Living services in the same way that such youth are included in the impact analysis. Thus, the costs and 
estimated benefits of the program, expressed in impacts, are directly comparable. Additionally, in this 
evaluation, the “cost per sample member” is very similar to the “cost per participant,” as 98.6 percent of 
individuals who were offered the program ultimately participated. 

34 All dollar amounts have been adjusted into 2015 dollars using the Consumer Price Index for all urban 
consumers. 
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Table 6 


Cost of Transitional Living Program Per Sample Member 


Cost Category 

Cost ($) 

Percentage of 
Total (%) 

Staff salaries and benefits 

8,952 

75.6 

Service providers 

7,082 

59.8 

Administrative staff 

1,869 

15.8 

Corporate overhead 

1,137 

9.6 

Travel 

1,088 

9.2 

Operations and maintenance 

396 

3.3 

Direct support to youth 

269 

2.3 

Total cost 

11,841 

100.0 

Sample size 


788 


SOURCES: Youth Village Transitional Living program expenditures and staffing schedules for fiscal 
years 20 1 1 -20 1 2 and 20 1 2-20 1 3 . 

NOTES: Costs have been adjusted to constant 2015 dollars using the Consumer Price Index for all 
urban consumers. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Total costs are divided by the number of program group members. 

The allocation of staff salaries and benefits to either "service providers" or "administrative staff' 
was based on the observation that 79.1 percent of the salaries in the staffing schedules were tied to 
"service provider" roles, including the following titles: TL Specialist, Clinical Supervisor, and 
Educational/Vocational Coordinator. The remaining 20.9 percent of the salaries in the staffing 
schedule were tied to "administrative staff', which included the following titles: Regional Supervisor, 
Regional Manager, Clinical Consultant, Compliance/Placement Specialist, Compliance Manager, 
Assistant Director, and Director. 


largely concentrated in areas that are difficult to monetize. For example, it is difficult to assign a 
dollar value to reductions in mental health problems and housing instability. Second, as will be 
discussed further in the next section, because Youth Villages is planning to implement program 
modifications intended to strengthen the program, it is premature to assess benefits and costs at 
this juncture.) 

The estimated gross cost per sample member of $1 1,841 should be interpreted in terms 
of the benefits accrued to the average program group member as a result of the offer of Transi- 
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tional Living services. Essentially, what does the cost of the program “buy” participants in 
improved outcomes? On average, based on all available data, including both Year 1 survey data 
and Year 2 administrative data, the program price tag of nearly $12,000 per sample member 
purchases modest increases in employment and earnings, reductions in homelessness and 
couch-surfing, reductions in experiences of economic hardship, improvements in mental health, 
and reductions in experiences of partner violence. These gains may also have resulted in 
benefits for society as a whole in the form of reductions in the use of homeless shelters and 
mental health services, among other benefits. However, the cost of the program does not 
purchase any improvements in the areas of education, criminal involvement, or social support. 
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Discussion and Policy Implications 


The Youth Villages Transitional Living Evaluation is one of the largest rigorous evaluations of 
services in the United States for young people who were formerly in the foster care or juvenile 
justice systems. The findings presented in this report on the two-year impacts of the Transitional 
Living program have important implications for future policymaking and research. This section 
considers these findings together with the one-year impacts of the program and discusses next 
steps for the Youth Villages Transitional Living program, now known as YVLifeSet. 


Impacts of the Transitional Living Program 

This evaluation has assessed the impacts of the Transitional Living program on outcomes in six 
important domains for young adults: education; employment and earnings; housing stability and 
economic well-being; social support; criminal involvement; and health and safety. As detailed 
in a previous report, one-year impacts in all six domains were estimated using survey data. This 
report builds on that analysis by estimating two-year impacts in three of the six domains — 
education, employment and earnings, and criminal involvement — using administrative data. 
Overall, the results show that the Transitional Living program led to modest, positive impacts 
on a broad range of outcomes in three of the six domains that were measured. 

• Survey and administrative data show that Transitional Living led to modest 
increases in employment and earnings. The evidence for this finding is 
stronger for Year 1 — the survey data showed a statistically significant im- 
pact on average earnings in Year 1; however, in both Year 1 and Year 2, the 
administrative data analysis indicates that the program increased the propor- 
tion of youth earning over $2,500 per year, a threshold that falls between the 
average annual earnings of the study sample in Years 1 and 2. 

• The analysis of the survey data also indicates that the program increased 
housing stability and economic well-being, including a reduction in home- 
lessness, by one year after study enrollment. Data are not available to assess 
whether impacts in that domain continued into the second year. 

• Similarly, the one-year analysis found significant impacts on some of the 
primary outcomes related to health and safety for these young people, but it 
is not known whether these impacts continued beyond one year because data 
were not available. 

• Both the one-year survey and two-year administrative data analyses indicate 
that Transitional Living did not lead to significant impacts in the remaining 
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three domains: education, social support (measured only at one year), or 
criminal involvement. The program did not lead to increases in high school 
graduation or receipt of a GED certificate at the one-year mark nor did it in- 
crease postsecondary enrollment over a two-year follow-up period. Similarly, 
it did not reduce criminal involvement as measured by self-reporting at one 
year or as measured by administrative data over a two-year period. Finally, 
there was not a significant impact on social support at the one-year mark. 

The impact analysis suggests that the Transitional Living program was able to improve 
outcomes related to immediate needs, such as having a place to live, being able to afford food 
and clothing, and avoiding violent relationships, but that the program was not as successful in 
affecting less-immediate outcomes, such as educational attainment. Given the level of disad- 
vantage of the young people served by the Transitional Living program and that many are 
experiencing a difficult transition from state custody to living independently, it may be that 
addressing the basic needs of participants requires a good deal of staff time and resources. 
Accordingly, less time may be left to address other issues. 

Nevertheless, while the lack of statistically significant impacts in some domains is dis- 
appointing, the impact findings as a whole are encouraging. Transitional Living succeeded in 
affecting a broad range of outcomes in some very important domains for young people who are 
experiencing the transition to adulthood. These findings are particularly noteworthy, given how 
few other programs have been shown to improve outcomes among young adults formerly in 
foster care or juvenile justice custody. While young people who have been in these systems, 
including those who received Transitional Living services, continue to face many challenges 
and to experience poor outcomes relative to their peers, the results of this study provide evi- 
dence that interventions are available that can lessen some of the difficulties that many of these 
young people face. 


Generalizability of the Research Findings Outside of Tennessee 

The findings presented in this report provide evidence about the effectiveness of the Transition- 
al Living program in the particular context of Tennessee, which is unusual in a number of 
respects. One contextual factor that may be important is the availability of other services. 
Community-based services for the foster care and juvenile justice populations were not widely 
available in Tennessee during the evaluation. In addition, the state’s extended foster care 
services were not generous or commonly accessed relative to some other states. The impacts of 
Transitional Living might be different if the program were implemented in another state that 
provides more extensive extended foster care services. In such a setting, impacts might be 
smaller, as the control group would have access to more services. On the other hand, with more 
extensive extended foster care services available, the Transitional Living program might be able 
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to focus less on housing and economic security and more on other areas, such as education, 
employment, and criminal involvement. If so, impacts in those other outcome domains might be 
larger than those that were found in this study. 

Another specific feature of the Tennessee context is that foster care and juvenile justice 
custody are the responsibilities of a single agency. This may mean that, given funding sources 
and recruitment avenues, the Transitional Living program would serve a different population of 
young people — most likely more exclusively from the foster care system — in other states in 
which juvenile justice custody is the responsibility of a separate agency. However, given that 
the findings presented in this report as well as previous reports provide little evidence of a 
difference in impacts by history of juvenile justice custody, the impacts of the Transitional 
Living program might be no different in a setting in which young people who have been 
involved with the juvenile justice system would be less likely to be recruited into the program. 

While this study provides evidence only about the effectiveness of the Transitional Liv- 
ing program in the particular context of Temiessee, it does also speak to the program’s effec- 
tiveness across a wide range of contexts within Tennessee. Contextual factors such as economic 
characteristics and the availability of resources and services vary substantially across the 
different regions and municipalities of the state. 35 Population density appeared to correlate with 
many of those factors. Compared with the more rural areas of the state, urban areas were 
characterized by a greater availability of social services, educational programs, and transporta- 
tion as well as more crime and gang activity. Despite such differences, there is very little 
evidence that the impacts of the Transitional Living program were different in urban areas 
compared with nonurban areas. This finding provides some indication that the impacts present- 
ed in this report may be applicable to other contexts in other states. 36 

Next Steps 

As an organization that emphasizes continuous learning and program improvement, Youth 
Villages is focused on using the evaluation findings as a springboard to strengthen the YVLife- 
Set model. Youth Villages plans to test new strategies intended to enhance the program’s 
positive effects on the young people it serves, building on the areas where the program has 
already been successful. Beginning in late 2016, Youth Villages will implement and test a few 
key modifications to the YVLifeSet model to assess whether they hold promise at a larger scale. 
Youth Villages is also considering launching a second large-scale study of YVLifeSet in a 
different context. 

35 Manno, Jacobs, Alson, and Skemer (2014). 

36 While Youth Villages does operate the Transitional Living program in other states that have other con- 
texts, the impacts of those programs have not been rigorously evaluated. Therefore, useful information is not 
available about whether the impacts of Transitional Living are similar in those contexts. 
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Appendix A 


Impacts by Subgroups of Young People 



Table A.l 


Two-Year Impacts on Selected Primary Outcomes, by History of Juvenile Justice Custody at Baseline 



Previously in Juvenile Justice Custody 

Never in Juvenile Justice Custody 

Difference 

Between 

Outcome 

Program 

Group 

Control 

Group 

Difference 

(Impact) 

P-Value 

Program 

Group 

Control Difference 

Group (Impact) 

P- Value 

Subgroup 

Impacts 3 

Education (%) 

Enrolled in a postsecondary institution 

15.4 

19.1 

-3.7 

0.160 

29.1 

32.4 

-3.3 

0.346 


Year 1 

13.0 

13.9 

-0.9 

0.698 

25.0 

23.5 

1.5 

0.638 


Year 2 

7.6 

8.9 

-1.2 

0.568 

15.9 

18.0 

-2.2 

0.481 


Employment and earnings 1 * ($) 

Total earnings 

4,540 

4,852 

-313 

0.549 

5,790 

5,399 

391 

0.501 


Year 1 

1,841 

1,966 

-125 

0.591 

2,582 

2,399 

183 

0.512 


Year 2 

2,699 

2,886 

-187 

0.614 

3,208 

3,000 

208 

0.587 


Criminal involvement (%) 

Arrested 

61.7 

55.0 

6.7 

* 0.078 

33.2 

39.3 

-6.1 

0.114 

tf 

Year 1 

43.2 

37.3 

5.9 

0.123 

18.1 

23.3 

-5.2 

0.119 

tt 

Year 2 

42.8 

41.6 

1.1 

0.775 

24.1 

28.7 

-4.6 

0.203 


Convicted 

26.8 

23.7 

3.1 

0.358 

12.4 

11.8 

0.6 

0.829 


Year 1 

14.9 

13.1 

1.8 

0.517 

5.8 

5.6 

0.3 

0.894 


Year 2 

19.0 

17.0 

2.1 

0.497 

7.5 

8.2 

-0.7 

0.771 


Sample size (total = 1,303) 

404 

272 



372 

255 





(continued) 



Table A.l (continued) 


SOURCE: MDRC calculations based on postsecondary education data from the National Student Clearinghouse, unemployment insurance 
quarterly earnings data from the Tennessee Department of Labor and Workforce Development, and criminal history data from the Tennessee 
Bureau of Investigation. 

NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model and adjusting for pre -random 
assignment characteristics. Impact estimates were then examined for statistically significant differences across subgroups. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess whether the difference in impacts 
between the subgroups is statistically significant. Statistically significant differences between subgroups are indicated as: f f f = 1 percent; f f = 5 
percent; f = 10 percent. 

b Unemployment insurance data are quarter-based; thus, the two-year follow-up period for employment and earnings outcomes does not 
necessarily represent the 24 months following random assignment. Rather, the follow-up period represents the eight quarters following the quarter 
in which participants were randomly assigned. 


4 ^ 

o 



Table A.2 


Two-Year Impacts on Selected Primary Outcomes, by Geographic Setting at Baseline 




Urban 



Nonurban 


Difference 

Between 

Outcome 

Program 

Group 

Control Difference 

Group (Impact) 

P-Value 

Program 

Group 

Control Difference 

Group (Impact) 

P- Value 

Subgroup 

Impacts 3 

Education (%) 

Enrolled in a postsecondary institution 

24.1 

29.6 

-5.4 * 

0.059 

20.1 

20.0 

0.1 

0.986 


Year 1 

20.2 

20.8 

-0.6 

0.800 

17.7 

15.5 

2.2 

0.450 


Year 2 

12.8 

17.6 

-4.8 * 

0.063 

10.0 

8.4 

1.6 

0.509 

f 

Employment and earnings'” ($) 

Total earnings 

5,038 

5,404 

-366 

0.463 

5,520 

4,635 

885 

0.137 


Year 1 

2,216 

2,326 

-109 

0.665 

2,254 

1,914 

341 

0.169 


Year 2 

2,822 

3,079 

-257 

0.436 

3,266 

2,721 

545 

0.200 


Criminal involvement (%) 

Arrested 

49.7 

52.1 

-2.4 

0.500 

45.4 

41.0 

4.4 

0.276 


Year 1 

32.9 

33.8 

-0.9 

0.802 

27.9 

27.4 

0.5 

0.901 


Year 2 

34.7 

37.6 

-2.9 

0.415 

33.2 

31.1 

2.0 

0.597 


Convicted 

18.6 

19.5 

-1.0 

0.741 

20.9 

15.4 

5.5 * 

0.091 


Year 1 

10.9 

11.2 

-0.4 

0.883 

9.4 

7.6 

1.7 

0.466 


Year 2 

12.0 

12.2 

-0.2 

0.936 

15.4 

12.9 

2.5 

0.408 


Sample size (total = 1,312) 

439 

295 



343 

235 
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Table A.2 (continued) 


SOURCE: MDRC calculations based on postsecondary education data from the National Student Clearinghouse, unemployment insurance quarterly 
earnings data from the Tennessee Department of Labor and Workforce Development, and criminal history data from the Tennessee Bureau of 
Investigation. 

NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model and adjusting for pre -random 
assignment characteristics. Impact estimates were then examined for statistically significant differences across subgroups. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess whether the difference in impacts 
between the subgroups is statistically significant. Statistically significant differences between subgroups are indicated as: f f f = 1 percent; f f = 5 
percent; f = 10 percent. 

b Unemployment insurance data are quarter-based; thus, the two-year follow-up period for employment and earnings outcomes does not necessarily 
represent the 24 months following random assignment. Rather, the follow-up period represents the eight quarters following the quarter in which 
participants were randomly assigned. 
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Table A.3 


Two-Year Impacts on Selected Primary Outcomes, by Receipt of EFC Services at Baseline 



Receiving EFC Services at Baseline 

Not Receiving EFC Services at Baseline 

Difference 

Between 

Outcome 

Program 

Group 

Control Difference 

Group (Impact) 

P-Value 

Program 

Group 

Control Difference 

Group (Impact) 

P-Value 

Subgroup 

Impacts 3 

Education (%) 

Enrolled in a postsecondary institution 

41.4 

53.6 

-12.1 ** 

0.046 

16.8 

18.4 

-1.6 

0.457 


Year 1 

35.1 

43.6 

-8.5 

0.121 

14.4 

12.4 

2.0 

0.297 

t 

Year 2 

21.3 

25.6 

-4.3 

0.449 

8.9 

10.3 

-1.4 

0.426 


Employment and earnings' 1 ($) 

Total Earnings 

6,145 

5,102 

1,043 

0.255 

4,973 

4,997 

-25 

0.954 


Year 1 

2,796 

1,661 

1,135 *** 

0.005 

2,084 

2,209 

-125 

0.535 

ttt 

Year 2 

3,349 

3,441 

-92 

0.891 

2,888 

2,788 

100 

0.724 


Criminal involvement (%) 

Arrested 

28.8 

34.5 

-5.6 

0.343 

52.8 

51.2 

1.6 

0.590 


Year 1 

12.5 

19.3 

-6.8 

0.151 

35.6 

34.4 

1.2 

0.674 


Year 2 

22.1 

24.3 

-2.2 

0.696 

37.2 

37.6 

-0.4 

0.882 


Convicted 

9.3 

8.5 

0.8 

0.837 

22.5 

20.1 

2.3 

0.353 


Year 1 

3.7 

4.5 

-0.7 

0.784 

12.2 

11.0 

1.2 

0.559 


Year 2 

5.8 

4.8 

0.9 

0.757 

15.6 

14.4 

1.2 

0.597 


Sample size (total = 1,322) 

180 

96 



608 

438 
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Table A.3 (continued) 


SOURCE: MDRC calculations based on postsecondary education data from the National Student Clearinghouse, unemployment insurance quarterly 
earnings data from the Tennessee Department of Labor and Workforce Development, and criminal history data from the Tennessee Bureau of 
Investigation. 

NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least squares model and adjusting for pre -random 
assignment characteristics. Impact estimates were then examined for statistically significant differences across subgroups. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 percent. 

a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is used to assess whether the difference in impacts 
between the subgroups is statistically significant. Statistically significant differences between subgroups are indicated as: f f f = 1 percent; ff = 5 
percent; f = 10 percent. 

b Unemployment insurance data are quarter-based; thus, the two-year follow-up period for employment and earnings outcomes does not 
necessarily represent the 24 months following random assignment. Rather, the follow-up period represents the eight quarters following the quarter in 
which participants were randomly assigned. 
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Table A.4 


Characteristics of Sample Members, by Latent Class Assignment at Baseline 


Characteristic 

Class l a 

Class 2 b 

Class 3 C 

Full 

Sample 

Age categories (%) 

18 years old 

74.4 

74.5 

65.7 

71.4 

19 years old 

22.1 

14.7 

22.2 

19.4 

20-24 years old 

3.5 

10.7 

12.2 

9.2 

Gender (%) 

Male 

67.0 

37.6 

55.7 

52.0 

Female 

33.0 

62.4 

44.3 

48.0 

Race/ethnicity (%) 

Hispanic 

5.2 

6.7 

5.4 

5.8 

White, non-Hispanic 

41.8 

55.7 

53.6 

51.1 

Black, non-Hispanic 

48.9 

32.7 

32.2 

37.1 

Other, non-Hispanic 

4.1 

4.9 

8.7 

6.0 

Ever employed (%) 

28.6 

49.7 

79.1 

54.1 

Employed at baseline (%) 

10.4 

18.0 

27.6 

19.2 

Educational attainment and school enrollment (%) 

No high school diploma or GED certificate, 

26.8 

14.1 

13.2 

17.3 

not enrolled in school 

No high school diploma or GED certificate, 

39.2 

47.9 

32.1 

40.0 

enrolled in school 

High school diploma or GED certificate, 

not enrolled in postsecondary school 

23.8 

24.4 

38.2 

29.1 

High school diploma or GED certificate, 

enrolled in postsecondary school 

10.2 

13.6 

16.5 

13.7 

Ever repeated a grade or been held back a grade (%) 

70.3 

40.5 

24.8 

43.3 

Ever been suspended from school (%) 

91.0 

66.0 

87.8 

80.5 

Ever been in special education (%) 

30.5 

22.8 

24.7 

25.6 


(continued) 
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Table A.4 (continued) 


Characteristic 

Class l a 

Class 2 b 

Class 3 C 

Full 

Sample 

Contact with biological mother (%) 

Every day 

70.6 

33.3 

31.7 

43.1 

At least once a week but not every day 

20.4 

14.9 

14.8 

16.4 

At least once a month but not every week 

2.2 

8.3 

11.5 

7.7 

Less than once a month 

1.6 

11.1 

10.0 

8.1 

Never 

5.2 

32.3 

32.0 

24.7 

Contact with biological father (%) 

Every day 

24.5 

13.3 

13.1 

16.4 

At least once a week but not every day 

17.7 

13.3 

9.2 

13.1 

At least once a month but not every week 

6.0 

8.5 

8.3 

7.7 

Less than once a month 

9.3 

12.5 

8.7 

10.3 

Never 

42.5 

52.3 

60.7 

52.5 

Had contact with any other relatives at least 

94.6 

85.3 

87.0 

88.4 

once per month (%) 

Ever arrested (%) 

100.0 

5.1 

100.0 

64.4 

Received psychological or emotional counseling 

56.1 

52.2 

59.1 

55.7 

in past year (%) 

Attended substance abuse treatment program 

44.5 

15.2 

37.6 

31.1 

in past year (%) 

State custody history 

Ever in state custody because of (%) 

Neglect, abuse, or unruly adjudication 11 (foster care) 

35.8 

79.5 

61.8 

61.3 

Delinquency (juvenile justice) 

76.0 

28.4 

58.0 

51.9 

Age in years at first custody entry (%) 

0-5 

2.2 

6.9 

9.4 

6.5 

6-10 

2.8 

8.7 

6.6 

6.3 

11-14 

17.7 

21.1 

29.8 

23.2 

15-16 

32.9 

31.5 

32.6 

32.3 

17-18 

44.5 

31.7 

21.7 

31.7 


(continued) 
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Table A.4 (continued) 


Characteristic 

Class l a 

Class 2 b 

Class 3 C 

Full 

Sample 

Age in years at final custody exit (%) 

16 or under 

2.5 

5.7 

5.7 

4.8 

17 

27.8 

29.0 

25.8 

27.6 

18 or over 

36.8 

40.0 

40.7 

39.4 

Still in custody at baseline 

32.9 

25.2 

27.8 

28.3 

Number of different custody placements (%) 

1 placement 

44.9 

37.4 

24.0 

34.8 

2-5 placements 

48.3 

51.3 

49.3 

49.8 

6-10 placements 

4.8 

7.2 

16.5 

9.8 

More than 1 0 placements 

2.0 

4.1 

10.1 

5.6 

Sample size 

367 

495 

460 

1,322 


SOURCE: MDRC calculations based on responses to the Youth Villages Transitional Living Evaluation 
Baseline Information Form. 

NOTES: The following variables were used to conduct latent class analysis: number of custody 
placements, ever employed, contact with mother or father at least once per week, ever repeated a grade 
or been held back, ever arrested, received psychological or emotional counseling or attended drug or 
alcohol abuse treatment in the past year. These variables were all created using information collected at 
baseline. 

GED = General Educational Development. 

Rounding may cause slight discrepancies in calculating sums and differences. 
a Class 1 represents the “Hindered but connected to family” latent class. 
b Class 2 represents the “Maltreated but avoiding trouble” latent class. 
c Class 3 represents the “Long-term system-involved but engaged” latent class. 
d An unruly adjudication occurs because children are determined to have behavioral problems serious 
enough that their health and safety are at risk or because they have committed an offense, such as 
truancy, that is applicable only to minors. 
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Table A.5 


Two-Year Impacts on Selected Primary Outcomes, 
by Latent Class Assignment at Baseline 


Outcome 

Program 

Group 

Control Difference 

Group (Impact) 

Difference 

Between 

Subgroup 
P-Value Impacts 3 

Hindered but connected to family 






Education (%) 






Enrolled in a postsecondary institution 

12.2 

16.5 

-4.2 

0.214 


Year 1 

9.5 

9.6 

-0.1 

0.977 


Year 2 

4.1 

9.7 

-5.6 ** 

0.041 


Employment and earnings ' 5 ($) 






Total earnings 

4,224 

3,035 

1,190 * 

0.053 


Year 1 

1,677 

1,336 

341 

0.266 


Year 2 

2,548 

1,699 

849 ** 

0.038 

f 

Criminal involvement (%) 






Arrested 

63.9 

56.6 

7.3 

0.175 


Year 1 

44.6 

39.2 

5.4 

0.321 


Year 2 

45.0 

43.5 

1.5 

0.789 


Convicted 

28.1 

25.2 

3.0 

0.553 


Year 1 

18.1 

16.3 

1.8 

0.676 


Year 2 

17.7 

17.7 

0.0 

0.998 


Sample size (total = 367) 

224 

143 




Maltreated but avoiding trouble 






Education (%) 






Enrolled in a postsecondary institution 

29.7 

31.3 

-1.7 

0.676 


Year 1 

25.0 

23.1 

1.9 

0.598 


Year 2 

17.7 

17.9 

-0.2 

0.956 


Employment and earnings ' 5 ($) 






Total earnings 

5,609 

5,089 

520 

0.417 


Year 1 

2,338 

2,197 

140 

0.644 


Year 2 

3,272 

2,892 

380 

0.393 

f 
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Table A.5 (continued) 


Outcome 

Program 

Group 

Control Difference 

Group (Impact) 

P- Value 

Difference 

Between 

Subgroup 

Impacts 3 

Criminal involvement (%) 

Arrested 

29.7 

32.9 

-3.2 

0.474 


Year 1 

15.6 

16.0 

-0.4 

0.903 


Year 2 

20.8 

25.5 

-4.7 

0.249 


Convicted 

9.2 

9.4 

-0.2 

0.940 


Year 1 

3.9 

2.7 

1.1 

0.530 


Year 2 

7.3 

7.2 

0.1 

0.977 


Sample size (total = 495) 

303 

192 




Long-term svstem-involved but engaged 

Education (%) 

Enrolled in a postsecondary institution 

21.2 

26.5 

-5.4 

0.152 


Year 1 

19.2 

21.1 

-1.9 

0.582 


Year 2 

10.2 

12.2 

-2.0 

0.510 


Employment and earnings ' 5 ($) 

Total earnings 

5,690 

6,344 

-653 

0.380 


Year 1 

2,580 

2,642 

-62 

0.854 


Year 2 

3,110 

3,701 

-591 

0.245 

f 

Criminal involvement (%) 

Arrested 

55.7 

54.1 

1.6 

0.733 


Year 1 

36.9 

39.1 

-2.2 

0.631 


Year 2 

39.8 

37.8 

2.0 

0.659 


Convicted 

24.6 

20.5 

4.2 

0.291 


Year 1 

11.2 

11.4 

-0.1 

0.962 


Year 2 

17.2 

13.6 

3.7 

0.288 


Sample size (total = 460) 

261 

199 
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Table A.5 (continued) 


SOURCE: MDRC calculations based on postsecondary education data from the National Student 
Clearinghouse, unemployment insurance quarterly earnings data from the Tennessee Department of 
Labor and Workforce Development, and criminal history data from the Tennessee Bureau of 
Investigation. 

NOTES: Program impacts were calculated separately for each subgroup, using an ordinary least 
squares model and adjusting for pre-random assignment characteristics. Impact estimates were then 
examined for statistically significant differences across subgroups. 

Rounding may cause slight discrepancies in calculating sums and differences. 

Statistical significance levels are indicated as follows: *** = 1 percent; ** = 5 percent; * = 10 
percent. 

a When comparing impacts between two subgroups, an H-statistic is generated. The H-statistic is 
used to assess whether the difference in impacts between the subgroups is statistically significant. 
Statistically significant differences between subgroups are indicated as: f f f = 1 percent; f f = 5 
percent; f = 10 percent. 

b Unemployment insurance data are quarter-based; thus, the two-year follow-up period for 
employment and earnings outcomes does not necessarily represent the 24 months following random 
assignment. Rather, the follow-up period represents the eight quarters following the quarter in which 
participants were randomly assigned. 
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MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to leam not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 



